Apparatus and method for building a pallet load

ABSTRACT

A pallet building apparatus for automatically building a pallet load of pallet load article units onto a pallet support including a frame defining a pallet building base, at least one articulated robot to transport and place the pallet load article units, a controller to control articulated robot motion and effect therewith a pallet load build, at least one three-dimensional, time of flight, camera to generate three-dimensional imaging of the pallet support and pallet load build, wherein the controller registers, from the three-dimensional camera, real time three-dimensional imaging data embodying different corresponding three-dimensional images of the pallet support and pallet load build, to determine, in real time, from the corresponding real time three-dimensional imaging data, a pallet support variance or article unit variance and generate in real time an articulated robot motion signal, the articulated robot motion signal being generated real time so as to be performed real time.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Patent Application No.17,151,761, filed Jan. 19, 2021, (now U.S. Pat. No. 11,691,830), whichis a continuation of United States Application No. 16/035,204, filedJul. 13, 2018, (now U.S Pat. No. 10,894,676), which is a non-provisionalof and claims the benefit of United States Provisional PatentApplication No. 62/533,503, filed on Jul. 17, 2017, the disclosures ofwhich are incorporated herein by reference in their entireties.

BACKGROUND 1. Field

The exemplary embodiments generally relate to storage and retrievalsystems and, more particularly, to palletizing/depalletizing cells ofthe storage and retrieval systems.

2. Brief Description of Related Developments

The retail distribution of products (whether for conventional “brick andmortar” stores, online stores, or mixed retail channels) demandsimprovements in storage, sortation, and transport efficiencies,particularly for distribution of what is known as mixed cases orheterogeneous cases (within a given transport) whether for storereplenishment or individual orders. The application ofintelligent/adaptive automation thereto has increasingly facilitatedimprovement in efficiency at many levels of distribution includingstorage, sortation and transport. Still, persistently difficult problemsremain producing bottlenecks that disrupt, or adversely impact generalsystem efficiency, such as the mixed case pallet load (or truck load)efficiency problem. As may be readily realized, the difficulty of thepallet load (or truck load) efficiency problem is not die singularlyfrom the desire for high packing density, but rather pallet loadefficiency is dependent on both packing density and building the palletload in a time optimal manner (i.e. the build puzzle of packing thepallet load to densities over 90% may be solved readily given whatevertime necessary and the necessary selection of mixed cases, but suchpallet load would not be efficient if the pallet load build time is nottime optimal).

Conventional pallet loaders (e.g., palletizers) and pallet unloaders(e.g., depalletizers) having electromagnetic radiation and opticalmapping sensors (e.g. laser scanners, 3-D cameras, etc.) so as to mapthe 3-D pallet load for improved automation positioning relative to thepallet lad are known. For example, one conventional method and systemfor detecting and reconstructing environments to facilitate roboticinteraction with such environments includes determining athree-dimensional (3-D) virtual environment where the 3-D virtualenvironment represents a physical environment of a robotic manipulatorincluding a plurality of 3-D virtual objects corresponding to respectivephysical objects in the physical environment. The method then involvesdetermining two dimensional (2-D) images of the virtual environmentincluding 2-D depth maps. The method may then involve determiningportions of the 2-D images that correspond to a given one or morephysical objects. The method may then involve determining, based on theportion and the 2-D depth maps, 3-D models corresponding to theportions. The method may then involve, based on the 3-D models,selecting a physical object from the given one or more physical objects.The method may then involve providing an instruction to the roboticmanipulator to move that object.

As another example, of a conventional method and system for detectingand reconstructing environments to facilitate robotic interaction withsuch environments includes the automatic determination of a model of apackage stack on a loading carrier (i.e., in particular pallets). Aninitial desired position for a package in the model is determined. Thepackage stack is detected on the loading carrier and a deviation betweenthe detected package stack and the model is determined. The package isplaced by an automated manipulator, and the above steps are repeateduntil a termination criterion is reached.

As may be realized from the representative examples, conventionalpalletizer(s) with 3-D mapping systems fail to provide truly adaptive,to near real time build variation, pallet load building automation so asto effect time optimal building of pallet loads.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing aspects and other features of the disclosed embodiment areexplained in the following description, taken in connection with theaccompanying drawings, wherein:

FIG. 1 is a schematic illustration of a distribution facility inaccordance with aspects of the disclosed embodiment;

FIG. 2 is a schematic illustration of a pallet load in accordance withaspects of the disclosed embodiment;

FIG. 3 is a schematic isometric view of a palletizer cell in accordancewith aspects of the disclosed embodiment;

FIG. 3A is a schematic exploded isometric view of the palletizer cell ofFIG. 3 in accordance with aspects of the disclosed embodiment;

FIG. 3B is a schematic plan or top view of the palletizer cell of FIG. 3in accordance with aspects of the disclosed embodiment;

FIG. 3C is a schematic right side view of the palletizer cell of FIG. 3in accordance with aspects of the disclosed embodiment;

FIG. 3D is a schematic front view of the palletizer cell of FIG. 3 inaccordance with aspects of the disclosed embodiment;

FIG. 3E is a schematic left side view of the palletizer cell of FIG. 3in accordance with aspects of the disclosed embodiment;

FIG. 3F is a schematic rear or back view of the palletizer cell of FIG.3 in accordance with aspects of the disclosed embodiment;

FIG. 3G is a schematic isometric view of the palletizer cell of FIG. 3in accordance with aspects of the disclosed embodiment;

FIG. 3H is a schematic left side view of the palletizer cell of FIG. 3in accordance with aspects of the disclosed embodiment;

FIG. 3I is a schematic front view of the palletizer cell of FIG. 3 inaccordance with aspects of the disclosed embodiment;

FIG. 3J is a schematic plan or top view of the palletizer cell of FIG. 3in accordance with aspects of the disclosed embodiment;

FIG. 3K is a schematic isometric view of the palletizer cell of FIG. 3showing, with emphasis, the field of view of a camera of a vision systemof the palletizer cell in accordance with aspects of the disclosedembodiment;

FIG. 3L is a schematic isometric view of the palletizer cell of FIG. 3showing, with emphasis, the field of view of a camera of a vision systemof the palletizer cell in accordance with aspects of the disclosedembodiment;

FIG. 3M is a schematic isometric view of the palletizer cell of FIG. 3showing, with emphasis, the field of view of a camera of a vision systemof the palletizer cell in accordance with aspects of the disclosedembodiment;

FIG. 3N is a schematic isometric view of the palletizer cell of FIG. 3showing, with emphasis, the field of view of a camera of a vision systemof the palletizer cell in accordance with aspects of the disclosedembodiment;

FIGS. 4, 4A and 4B are illustrations of pallet supports, disposed on apallet building base of the palletizer cell of FIG. 3 , generated fromreal time three dimensional imaging data (e.g. point cloud data) from avision system of the palletizer cell where defects or variances in thepallet support are detected in accordance with aspects of the presentdisclosure;

FIG. 4C is a schematic illustration of a pallet support in accordancewith aspects of the present disclosure;

FIGS. 5 and 5A-5G illustrate a sequence of real time three-dimensionalimaging data (e.g., point cloud data) from a vision system of apalletizer cell corresponding to a pallet build where a tilted case unitis detected by the vision system upon placement of the case unit(s) inaccordance with aspects of the present disclosure;

FIGS. 6 and 6A-6H illustrate a sequence of real time three-dimensionalimaging data (e.g., point cloud data) from a vision system of apalletizer cell corresponding to a pallet build where a fallen case unit(e.g. the case unit has fallen on the floor) is detected by the visionsystem upon placement of the case unit(s) in accordance with aspects ofthe present disclosure;

FIGS. 7 and 7A-7F illustrate a sequence of real time three-dimensionalimaging data (e.g., point cloud data) from a vision system of apalletizer cell corresponding to a pallet build where a fallen case unit(e.g. the case unit has fallen on the floor) is detected by the visionsystem upon placement of the case unit(s) in accordance with aspects ofthe present disclosure;

FIGS. 8 and 8A-8G illustrate a sequence of real time three-dimensionalimaging data (e.g., point cloud data) from a vision system of apalletizer cell corresponding to a pallet build where a fallen case unit(e.g. the case unit has fallen on the pallet) is detected by the visionsystem upon placement of the case unit(s) in accordance with aspects ofthe present disclosure;

FIGS. 9 and 9A-9E illustrate a sequence of real time three-dimensionalimaging data (e.g., point cloud data) from a vision system of apalletizer cell corresponding to a pallet build where a fallen case unit(e.g. the case unit has fallen from above the pallet onto the pallet orpallet building base) is detected by the vision system upon placement ofthe case unit(s) in accordance with aspects of the present disclosure;

FIGS. 10 and 10A-10D illustrate a sequence of real timethree-dimensional imaging data (e.g., point cloud data) from a visionsystem of a palletizer cell corresponding to a pallet build where afallen case unit (e.g. the case unit has fallen from above the palletonto the pallet or pallet building base) is detected by the visionsystem upon placement of the case unit(s) in accordance with aspects ofthe present disclosure;

FIG. 11 is a flow diagram in accordance with aspects of the presentdisclosure; and

FIG. 12 is a flow diagram in accordance with aspects of the presentdisclosure.

DETAILED DESCRIPTION

FIG. 1 is a schematic illustration of a warehouse system or distributionfacility 100WS (referred to herein as warehouse system 100WS) inaccordance with aspects of the disclosed embodiment. Although theaspects of the disclosed embodiment will be described with reference tothe drawings, it should be understood that the aspects of the disclosedembodiment can be embodied in many forms. In addition, any suitablesize, shape or type of elements or materials could be used. It should beunderstood that while the distribution facility 100WS is describedherein as an automated distribution facility the aspects of thedisclosed embodiment are also applicable to distribution facilitieshaving any suitable transport systems, such as both automated and manualtransport systems or to wholly manual transport systems.

Referring to FIGS. 1 and 2 , in accordance with the aspects of thedisclosed embodiment, the warehouse system 100WS includes at least onereal time adaptive palletizer/depalletizer cell 10A, 10B (generallyreferred to herein as palletizer cell 10). The palletizer cell 10 hasone or more robotic case manipulator(s) 14 (also referred to herein asarticulated robots or robots) that place (individually or manufacturedpickfaces) mixed pallet load article units CU (also referred to hereinas case units or cases) in stacks SL1-Sn and/or layers PL1-PL4 buildinga mixed case pallet load PAL with vision system assistance.

The palletizer cell 10 is provided with a three-dimensional (3D) time offlight (TOF) camera(s) vision system 310 (referred to herein as thevision system 310), that generates 3D imaging of each case unit CUplacement, by the robot 14, the pallet load build (on the palletsupport) BPAL, and of the pallet support SPAL. The three-dimensionalimage information is generated and provided by the vision system 310, inreal time coincident with robot 14 cyclic motion placing case units CUbuilding the pallet load PAL and informs in real time (within the robot14 place motion cycle frame), place pose of each placed case unit CU androbot placement pose of each following case unit CU in the pallet buildfrom first case layer PL1 seated on the pallet support SPAL to the lastcase layer PL5.

The place pose three-dimensional image information of each case unit CU,and of the whole/part pallet load build BPAL, and of the pallet supportSPAL, identifies variances from plan, that inform compensation for thevariances to, for example, the robot 14 so that the robot 14 compensateswith subsequent robot 14 case unit CU placement pose or other palletbuild response in real time, so as to facilitate substantiallycontinuous, with adaptive real time case placement, and adaptive palletbuild (in full automation or in collaboration/cooperation with userassist) and coincidently resolve pallet quality/controls and build withthe robot 14.

The vision system 310, incorporated into the automated palletizer cell10, informs and enables a cell controller 10C so as to provide, realtime command inputs (to the automation such as the robot(s) 14) that areresponsive, in real time to pallet load building variances so that therobot(s) 14 is adaptive in real time resolving pallet load buildvariances, affecting pallet build, (automatically and/or incooperation/collaboration with user assistance) in time optimal mannerso as to effect the pallet load build in time optimal manner. Theadaptive pallet cell automation, facilitated by the real time visionsystem assistance, is also responsive to identify and correct deviantpallet build conditions (automatically and/or in cooperation/collaboration with user assist) obstructing or impeding time optimalpallet load build.

Referring again to FIG. 1 , in accordance with aspects of the disclosedembodiment the distribution facility 100WS includes a storage andretrieval system 100 that may operate in a retail distribution center orwarehouse to, for example, fulfill orders received from retail storesfor case units. In one example, the case units may be cases or units ofgoods not stored in trays, on totes or on pallets (e.g. uncontained). Inother examples, the case units may be cases or units of goods that arecontained in any suitable manner such as in trays, on totes or onpallets. It is noted that the case units may include cased units ofgoods (e.g. case of soup cans, boxes of cereal, etc.) or individualgoods that are adapted to be taken off of or placed on a pallet. Inaccordance with the embodiments, shipping cases for case units (e.g.cartons, barrels, boxes, crates, jugs, or any other suitable device forholding case units) may have variable sizes and may be used to hold caseunits in shipping and may be configured so they are capable of beingpalletized for shipping. It is noted that when, for example, bundles orpallets of case units arrive at the storage and retrieval system thecontent of each pallet may be uniform (e.g. each pallet holds apredetermined number of the same item - one pallet holds soup andanother pallet holds cereal) and as pallets leave the storage andretrieval system the pallets may contain any suitable number andcombination of different case units (e.g. each pallet may hold differenttypes of case units - a pallet holds a combination of soup and cereal).In the embodiments the storage and retrieval system described herein maybe applied to any environment in which case units are stored andretrieved.

The storage and retrieval system 100 may be configured for installationin, for example, existing warehouse structures or adapted to newwarehouse structures. In the aspects of the disclosed embodiment, thestorage and retrieval system may include one or more in-feed transferstation 170 and one or more out-feed transfer station 160, in/out caseconveyors 150A, 150B, 150C (generally referred to as in/out caseconveyors 150), a storage structure array 130, and a number ofautonomous vehicular transport robots 110 (referred to herein as“bots”). In the aspects of the disclosed embodiment the storage andretrieval system may also include robot or bot transfer stations, asdescribed in U.S. Pat. No. 9,096,375 issued on Aug. 4, 2015 thedisclosure of which is incorporated by reference herein in its entirety.In the embodiments the bot transfer stations may provide an interfacebetween the bots 110 and the in/out case conveyors 150 such that caseunits can be indirectly transferred between the bots 110 and the in/outcase conveyors 150 through the bot transfer stations. In the embodimentscase units may be transferred directly between the bots 110 and thein/out case conveyors 150.

The storage structure array 130 may include multiple levels of storagerack modules that form a storage array of storage locations 130SL forcase units, each storage location 130SL of which is arranged for storageof at least one case unit at each storage location 130SL. In one aspect,each level of the storage structure array 130 includes respectivestorage/picking aisles 130A, and transfer decks 130B for transferringcase units between any of the storage areas of the storage structurearray 130 and any shelf of any in/out case conveyors 150. The storageaisles 130A, and transfer decks 130B are also configured to allow thebots 110 to traverse the storage aisles 130A and transfer decks 130B forplacing case units into picking stock and to retrieve ordered caseunits, where the case units are stored or otherwise held in the storageaisles 130A and/or on the transfer deck 130B in storage locations 130SL.The bots 110 may be any suitable bots capable of carrying andtransferring case units throughout the storage and retrieval system 100.Suitable examples of bots can be found in, for exemplary purposes only,U.S. Pat. No. 8,425,173 issued on Apr. 23, 2013, U.S. Pat. No. 9,561,905issued on Feb. 7, 2017, U.S. Pat. No. 8,965,619 issued on Feb. 24, 2015,U.S. Pat. No. 8,696,010 issued on Apr. 15, 2014, U.S. Pat. No. 9,187,244issued on November 113/326,952 (which is non-provisional of U.S. SerialNo. 61/423,365 filed on Dec. 15, 2010) entitled “Automated Bot withTransfer Arm” filed on Dec. 15, 2011, and U.S. Pat. No. 9,499,338 issuedon Nov. 22, 2016, the disclosures of which are incorporated by referenceherein in their entireties. The bots 110 may be configured to place caseunits, such as the above described retail merchandise, into pickingstock in the one or more levels of the storage structure array 130 andthen selectively retrieve ordered case units for shipping the orderedcase units to, for example, a store or other suitable location.

The in-feed transfer stations 170 and out-feed transfer stations 160 mayoperate together with their respective in/out case conveyors 150A, 150Bfor bi-directionally transferring case units to and from one or morelevels of the storage structure array 130 effecting infeed of the caseunits into the storage structure array 130 and output of the case unitsfrom the storage structure array 130. It is noted that while the in-feedtransfer stations 170 and the outfeed transfer stations 160 (and theirrespective in/out case conveyors 150A, 150B and palletizer/depalletizercells 10A, 10B) are described as being dedicated inbound (e.g. in-feed)transfer stations 170 and dedicated outbound (e.g. out-feed) transferstations 160, in the aspects of the disclosed embodiment each of thetransfer stations 170, 160 may be used for both inbound and outboundtransfer of case units from the storage and retrieval system. It isnoted that while in/out case conveyors are described herein, theconveyors may be any suitable conveyors (including any suitabletransport path orientation, such as vertical and/or horizontal conveyorpaths) or transfer/picking devices having any suitable transport pathorientation.

In one aspect, as described above, each of the in-feed transfer stations170 and the out-feed transfer stations 160 include a respective in/outcase conveyor 150A, 150B and a respective palletizer/depalletizer cell10A, 10B (referred to generally herein as palletizer cell 10). In oneaspect, the palletizer/depalletizer cells 10 are automated cells eachbeing configured to receive loaded pallets (such as with uniform ormixed case units or products) from, for example, a pallet load in 175area which may include an in-out loaded pallet conveyor 175C(illustrated in FIG. 1 as an input conveyor) and/or build a loadedpallet (such as with uniform or mixed case units or products) fortransport to, for example, a pallet load out 180 area which may includean in-out loaded pallet conveyor 180C (illustrated in FIG. 1 as anoutput conveyor). In one aspect, the conveyors 175C, 180C are eachconnected to the storage structure array 130 and are configured so as tobi-directionally transport loaded pallets in an input direction towardsthe storage structure array 130, and in a different output directionaway from the storage structure array 130. In one aspect, the conveyors175C, 180C may each include a conveyor arrangement with a distributedconveyor bed arranged to form a conveying path or in other aspects, theconveyors 175C, 180C may be discrete transport units such as, forexample, a fork lift/pallet truck. Suitable examples of automatedpalletizer/depalletizer cells 10A, 10B may be found in U. S. Pat.application No. 15/235,254 filed on Aug. 12, 2016, and U.S. Pat. No.8,965,559 issued on Feb. 24, 2015, the disclosures of which areincorporated herein by reference in their entireties. Each palletizercell includes one or more robotic case manipulators 14, which may alsobe referred to articulated robots or robots. The one or more roboticcase manipulators 14 are configured, as described herein, so as totransport and place the pallet load article units CU (also referred toherein as cases or case units) serially onto a pallet support so as tobuild the pallet load 250 on a pallet building base 301 (see FIG. 3 ).

Where the palletizer cell 10 functions in an output role as apalletizer, pallet load article units CU, that can be of various sizes,arrive at the palletizer cell 10 via the in/out case conveyors 150B, arepicked by one of the robotic case manipulators 14 and placed on thepallet PAL as will be described herein. Where the palletizer cell 10functions in an output role as a palletizer, a full pallet PAL (see FIG.2 ) made from a variety of case units is ready to be picked up by aforklift from the palletizer cell 10 for conveyance to a pallet load out180 area. Where the palletizer/depalletizer cell 10 functions in aninput role as a depalletizer, a full pallet (which may be similar topallet PAL and formed of homogenous or mixed cases) made from a varietyof pallet load article units CU is transferred to the palletizer cell 10in any suitable manner, such as a fork lift, from a pallet load in 175area. The one or more robotic case manipulators 14 pick the pallet loadarticle units CY from the pallet PAL for transfer into the storagestructure array 130.

In one aspect, each in-feed transfer station 170 forms, a case inputpath Ip where the palletizer/depalletizer cell 10A depalletizes caseunits, layer by layer, or otherwise depalletizes the case units intosingle case units from standard pallets (e.g. homogenous pallets havinga stability suitable for automatic engagement of a pallet layer by anautomatic layer interface unit, such as the product picking apparatus14). The palletizer/depalletizer cell 10A is in communication with atransport system of the automated storage and retrieval system 100, suchas an in/out case conveyor 150A so as to form an integral input system(e.g. the in-feed transfer station 170) that feeds case units to theautomated storage and retrieval system 100. Each in-feed transferstation 170 defines the case input path Ip that is integrated with theautomated storage and retrieval system 100 and warehouse managementsystem 199, where the warehouse management system 199 includes anysuitable controller 199C configured with any suitable non-transitoryprogram code and memory to manage, at least, case unit input to thestorage structure array 130B, case unit storage distribution within thestorage structure array 130B and case unit retrieval from the storagestructure array 130B, case unit inventory/replenishment and case unitoutput.

In one aspect, each case unit input path Ip includes at least onecorresponding case unit inspection cell 142 in communication with thewarehouse management system 199. In one aspect, the at least onecorresponding case unit inspection cell 142 may be any suitableinspection cell including any suitable volumetric inspection, such aswith a multi-dimensional light curtain, imaging systems and/or any othersuitable sensing/sensor arrangement configured to detect case unitdefects and identify the case units for, e.g., inventory, transportsequencing, storage distribution and sequencing the case unit for outputfrom the storage structure array 130B.

In one aspect, as noted above, the palletizer/depalletizer cell 10A maybe fully automatic so as to break down or decommission layer(s) from apallet unloading at the palletizer/depalletizer cell 10A. It is notedthat, referring to FIG. 2 , the term decommission refers to the removalof a pallet layer PL1, PL2, PL3, PL4 (in whole or in part) from a palletPAL so that each pallet load article unit CU is removed from the layerPL1, PL2, PL3, PL4 at a predetermined level 200 (which may correspond toa decommissioning/commissioning level or transfer plane) of the palletPAL so that the pallet PAL is indexed to a next level of the pallet PALfor removal of the next layer PL2, PL3 (in whole or in part)corresponding to the next level of the pallet PAL.

In one aspect, the palletizer/depalletizer cell 10A is configured todecommission the layers PL1, PL2, PL3, PL4 so that the decommissioningis synchronous or otherwise harmonized (e.g. matched with) by thewarehouse management system 199 with a predetermined rate of case unitflow or feed rate, established by the warehouse management system 199,in the automated storage and retrieval system 100. For example, in oneaspect, the warehouse management system 199 is configured to set and/ormonitor a predetermined rate of case unit flow within the automatedstorage and retrieval system 100. For example, the warehouse managementsystem 199 monitors and manages the automated systems of the automatedstorage and retrieval system 100 (such as, e.g., the in/out caseconveyors 150A, 150B, bots 110 and palletizer/depalletizer cells 10A,10B), where each of the automated systems, or one or more of automatedsystems have a given transaction time (such as a time/period to effect abasic unit of transport or transfer of cases, e.g. to transfer a caseunit on/off the in/out case conveyor to a pick/place station, or lift acase unit a predetermined distance, or bot transfer pick/place on astorage location, a time to transfer a pallet layer to or from a pallet,etc.) that in effect, singularly or in combination define, under controlof the warehouse management system 199 or any other suitable controllerof the automated storage and retrieval system 100 (e.g. bot controllers,conveyor controllers, palletizer/depalletizer controllers, etc.), thepredetermined rate of case unit flow in the automated storage andretrieval system 100 established by the warehouse management system 199.For example, the controller 199C of the warehouse management system 199is communicably connected to the in-out case conveyor(s) 150A, 150B sothat the in-out case conveyor(s) 150A, 150B bi-directionally transportthe case units to and from the storage structure array 130 at apredetermined case feed rate. The controller 199C may also becommunicably connected to a palletizer-depalletizer cell 10A, 10Bcorresponding to the in-out case conveyor(s) 150A, 150B so that thelayer commissioning and decommissioning of the palletizer/depalletizercell 10A, 10B, which are respectively substantially continuous, matchesthe predetermined case feed rate. While the aspects of the disclosedembodiment are described herein with respect to a distribution facility100WS having automated storage and retrieval system 100 with automatedtransport systems, the aspects of the disclosed embodiment are alsoapplicable to distribution facilities having any suitable transportsystems such as both automated and manual transport systems or to whollymanual transport systems, where both the automated transporttransactions and the manual transport transactions each have respectivetransaction times where the commissioning and decommissioning of caseunits to and from pallets may be matched to the transaction times in amanner substantially similar to that described herein.

In one aspect, each out-feed transfer station 160 forms, a case outputpath Op where the palletizer/depalletizer cell 10B palletizes caseunits, layer by layer onto pallets PAL such as with an automatic layerinterface unit, such as the one or more robotic case manipulators 14. Inone aspect, the pallets PAL may be formed as standard pallets (e.g.homogeneous case units) or as mixed pallets, such as described U.S. Pat.No. 14/997,920 filed on Jan. 18, 2016 the disclosure of which isincorporated herein by reference in its entirety. In one aspect, thewarehouse management system 199 is configured to establish a palletsolution, with mixed case units, that provides a stable pallet loadstack suitable for an end effector of the one or more robotic casemanipulators 14 to transfer as a layer. As described above, a suitableexample, of the palletizer/depalletizer cell 10B may be found in U.S.Pat. Application No. 15/235,254 filed on Aug. 12, 2016, the disclosureor which was previously incorporated herein by reference in itsentirety.

In one aspect, the palletizer/depalletizer cell 10B is in communicationwith a transport system of the automated storage and retrieval system100, such as an in/out case conveyor 150B so as to form an integraloutput system (e.g. the out-feed transfer station 160) that receivescase units from the automated storage and retrieval system 100 forplacement on pallets according to any suitable case out order sequence.For example, as described above, pallet load article units CU routed tothe one or more robotic case manipulators 14 are transferred to thepallet PAL by the end effector of the one or more robotic casemanipulators 14, with the pallet load article units CU (output caseunits) being arranged in a predetermined sequence established by thewarehouse management system 199, layer by layer (noting that the layermay cover the pallet in whole or in part) to form a standard outputpallet load.

Each out-feed transfer station 160 defines the case output path Op thatis integrated with the automated storage and retrieval system 100 andwarehouse management system 199, where the warehouse management system199 includes any suitable controller 199C configured with any suitablenon-transitory program code and memory to manage the operation of thedistribution facility 100WS, including case unit output from the storagestructure array 130B, as described herein. In one aspect, each case unitoutput path Op includes at least one corresponding case unit inspectioncell 142 (as described above) in communication with the warehousemanagement system 199. In one aspect, as noted above, thepalletizer/depalletizer cell 10B may be fully automatic so as to buildor commission layer (s) to a pallet loading at thepalletizer/depalletizer cell 10B. It is noted that, referring to FIG. 2, the term commission refers to the construction of a pallet layer PL1,PL2, PL3, PL4 (in whole or in part) to a pallet PAL so that each palletload article unit CU is inserted to the layer PL1, PL2, PL3, PL4 at apredetermined level 200 (which may correspond to adecommissioning/commissioning level or transfer plane) of the pallet PALuntil the pallet layer PL1, PL2, PL3, PL4 is formed so that the palletPAL is indexed to a next level of the pallet PAL for building of thenext layer PL1, PL2 (in whole or in part) corresponding to the nextlevel of the pallet PAL. In one aspect, the palletizer/depalletizer cell10B is configured to commission the layers PL1, PL2, PL3, PL4 so thatthe commissioning is synchronous or otherwise harmonized (e.g. matchedwith) by the warehouse management system 199 with a predetermined rateof case unit flow or feed rate, established by the warehouse managementsystem 199, in the automated storage and retrieval system 100 in amanner substantially similar to that described above with respect to thedecommissioning of the layers PL1, PL2, PL3, PL4 where the warehousemanagement system 199 manages case unit retrieval order and the sequenceof mixed case unit output to loadout sequence of the mixed case unitpallet load, and other associated aspects of output such as inventoryreconciliation.

Referring now to FIGS. 1, 3 and 3A-3F, the palletizer cell(s) 10 (it isnoted that the term “palletizer” is used for its convenience, and asnoted above, the features of the palletizer may also be effected in adepalletizer as otherwise applicable) is coupled to the storage andretrieval system 100 so as to communicate case unit CU (see FIG. 2 )flow (see the case output path(s) Op and the case input paths(s) Ip)with the storage retrieval system 100. The palletizer 10 is, inaccordance with aspects of the disclosed embodiment, an adaptivepalletizer system 300 that effects time optimal pallet load build andthus may compliment and leverage the storage and retrieval system 100case order flow throughput (though in other aspects the adaptivepalletizer 300 may be coupled to any suitable storage and retrievalsystem including conventional, manual, or semi-automated retrievalsystem with manually loaded feed station for the palletizer 10).

Referring also to FIG. 2 , the palletizer cell(s) 10 are configured tobuild pallet loads PAL where the pallets load PAL have a pallet loadbuild structure RPAL (system features may also be similarly applied totruck load) that is a three-dimensional array, structured in stacksS1-Sn and layers PL1-PL5, of mixed case (s) or pallet load article unitsCU including manufactured/constructed article units (pickfaces) each ofmultiple cases/articles placed onto the pallet / pallet support SPAL(case units / pallet load article units means case, tote, pack, shrinkwrap, etc). The pallet load build structure RPAL is determined bycontrol from ordered case unit(s) CU (e.g. case units CU output from thestorage and retrieval system 100). For example, in one aspect, apalletizer controller 10C may be coupled to the controller 199C of thewarehouse management system 199; while in other aspects, the palletizercontroller 10C may form a module of an integrated warehouse managementcontroller managing conveyance of the storage and retrieval system 100components including palletizer/depalletizer cell(s) 10, so as toreceive the information defining the pallet load build structure RPALincluding corresponding datum reference bounds, case pose and variancethreshold from references for the pallet load build effected by thepalletizer 10. The case pose sequence, in which the robot(s) 14 of thepalletizer 10 build the pallet load PAL may be effected by the storageand retrieval system 100 so cases output by the storage and retrievalsystem 10 feeding the bot pick station 350 of the palletizer 10 arrive(just in time or suitably buffered) in the predetermined pick sequencefor building the pallet load PAL, enabling a higher pick/place rate ofthe robot(s) 14 (e.g., the output case flow from the storage andretrieval system 100 substantially eliminates or reduces case unit CUsortation with the robot(s) 14). Suitable examples of output case flowsortation from the storage and retrieval system 100 can be found in, forexample, U.S. Publication Nos. US2016/0214808 published on Jul. 28,2016; US2016/0207709 published on Jul. 21, 2016; US2016/0207711published on Jul. 21, 2016; US2016/0214797 published on Jul. 28, 2016;US2016/0167880 published on Jun. 16, 2016; and US2016/0207710 publishedon Jul. 21, 2016, the disclosures of which are incorporated herein byreference in their entireties. Robot 14 pick/place rate for example hasa pick/place cycle, from pick at the input station (e.g. the bot pickstation 350) to place on pallet load build BPAL and return, of about 5sec. (with 2 robots the pick/place cycle is about 2.5 secs), and anadaptive feedback loop FBL (see FIG. 3 ) of the vision system 310 iseffected within the pick/place cycle of the robot(s) 14, in real time,so as to effect substantially continuous build of the pallet load buildstructure RPAL.

Referring now to FIGS. 3 and 3A-3F, each palletizer cell 10 generallyincludes a frame 300F, at least one robot 14, a controller 10C, and avision system 310 including at least one three-dimensional, time offlight, camera 310C. The frame 300F defines a pallet building base 301for the pallet support SPAL (FIG. 2 ). The at least one robot 14 isconnected to the frame 300F and is configured so as to transport andplace the pallet load article units CU (see also FIG. 2 ) serially ontothe pallet support SPAL (see FIG. 2 ) so as to build the pallet load PAL(see FIG. 2 ) on the pallet building base 301. The controller 10C isoperably connected to the at least one robot 14 and is configured (withany suitable hardware and non-transient computer program code) tocontrol articulated robot motion, relative to the pallet building base301, and effect therewith a pallet load build BPAL of the pallet loadPAL.

The at least one three-dimensional, time of flight, camera 310C of thevision system 310 is disposed on one or more of the frame 300F and therobot(s) 14 so as to generate three-dimensional imaging (e.g., 3D images500-507, 600-608, 700-706, 800-807, 900-905, 1000-1004 - see FIG. 5-10D,where each image in the respective series of images may be sequentiallygenerated upon placement of each case unit CU in the pallet load buildBPAL to identify variances in the pallet load build BPAL as describedherein) of the pallet support SPAL on the pallet building base 301 andof the pallet load build BPAL on the pallet support SPAL. While the atleast one three-dimensional camera 310C is descried herein as a time offlight camera, any suitable three-dimensional sensor/imager may be usedincluding laser scanners, sonar or other suitable machine visionsystems. As described herein, the at least one three-dimensional camera310C is communicably coupled to the controller 10C so the controller 10Cregisters, from the at least one three-dimensional camera 310C, realtime three-dimensional imaging data (such as the point cloudsillustrated in FIG. 5-10D and/or any suitable data obtained from thepoint clouds) embodying different corresponding three-dimensional imagesof the pallet support SPAL and of each different one of the pallet loadarticle units CU, and of the pallet load build BPAL being built on thepallet support SPAL.

In one aspect, the at least one three-dimensional camera 310C isconfigured so as to effect three-dimensional imaging of the palletsupport SPAL on the pallet building base 301 and of the pallet loadbuild BPAL on the pallet support SPAL with the at least one articulatedrobot 14 effecting substantially continuous pick/place cycles from theinput station (such as pick station 350) and placing each of the palletload article units CU building the pallet load PAL on the palletbuilding base 301. In one aspect, the at least one three-dimensionalcamera 310C is configured so as to effect three-dimensional imaging ofeach respective pallet load article unit CU substantially coincidentwith placement of the respective pallet load article unit CU by the atleast one articulated robot 14 effecting substantially continuouspick/place cycles from the input station (such as pick station 350) andplacing the pallet load article unit CU building the pallet load buildBPAL substantially continuously.

In one aspect, the at least one three-dimensional camera 310C includesfour (4) cameras 310C1, 310C2, 310C3, 310C4 (FIG. 3A) coupled to theframe 300F in any suitable locations so that the cameras 310C1, 310C2,310C3, 310C4 each have a respective field of view FOV1-FOV4 (FIG. 3A)for imaging at least two sides, e.g., a top (see FIG. 2 ) and one of afront side surface, a rear side surface and a vertical side surface(extending between the front and rear) (see FIG. 2 ) of the palletsupport SPAL and pallet load build BPAL / pallet load build structureRPAL. The at least one camera 310C may be oriented so that the top andat least one side surface (e.g. front, rear or a vertical side) of thepallet support SPAL and of each case unit CU placed on the palletsupport SPAL is visible within the field of view FOV1-FOV4 covering acorresponding portion of the pallet support SPAL / pallet load buildstructure RPAL. Referring also to FIGS. 3G-3J, in one aspect the cameras310C1, 310C2, 310C3, 310C4 may have any suitable focal length for apredetermined image intensity and be placed at, for example, a 45° angle(see FIG. 3H) relative to the frame 300F (e.g. such as a horizontalplane of the frame 300F as defined by, for example, the pallet buildingbase 301) and/or each other so that the at least two sides are imaged bythe at least one camera; while in other aspects, the angle between thecameras 310C1, 310C2, 310C3, 310C4 and/or the frame 300F may be more orless than 45°. In one aspect, each field of view FOV1-FOV4 (generallyreferred to as field of view FOV (see FIG. 3H and FIGS. 3K-3N whichillustrate each of the fields of view with emphasis relative to theother fields of view) of the cameras 310C1, 310C2, 310C3, 310C4 may be a45° field of view; while in other aspects the field of view FOV may bemore or less than 45° so long as at least two sides of the palletsupport SPAL and of the pallet support SPAL and pallet load build BPAL /pallet load build structure RPAL are imaged.

In one aspect, the at least one camera 310C resolves three-dimensionaldefinition of case unit features (e.g., edges of the case units) fromtwo or more orthogonal planes so that a maximum certainty of featurepose (e.g., the X, Y, Z, θ, α, µ positions of the feature - see FIG. 3G)is obtained from a single image of items in the respective field(s) ofview FOV1-FOV4 of the at least one camera 310C. Here the resolution ofthe three-dimensional definition of case unit features is independent ofcamera 310C placement (so long as the top and one side are imaged) andis performed in real time (e.g. within the pick/place cycle of the atleast one robot 14).

While four (4) cameras 310C1-310C4 are described, it should beunderstood that more or less than four (4) cameras 310C may be used andplaced so that the field of view of the camera(s) 310C of the visionsystem 310 cover(s) the pallet building base 301 of the frame 300F, apallet support SPAL seated on the pallet building base 301 and a whole(or at least a predetermined part) of the expected pallet load buildstructure RPAL, so as to capture, with any suitable desired resolution,three-dimensional time of flight images of object(s) desirablyeverywhere on the pallet support SPAL, and everywhere on the pallet loadbuild structure RPAL. The combined field(s) of view FOV1-FOV4 result insubstantially complete 360° coverage of the pallet load build structureRPAL with overlap of the field(s) of view FOV1-FOV4. For example, thecombined field(s) of view FOV1-FOV4 may cover standard pallet supportsSPAL (having dimensions of, e.g., 48 inches by 48 inches, 48 inches by40 inches, and/or 36 inches by 36 inches), it should be understood thatthe camera(s) 30Ca-300C4 and associated field(s) of view FOV1-FOV4 maycover (e.g. image) larger fields (including, for example, truck beds orany desired field size) as appropriate. Further, the field(s) of viewFOV1-FOV4 may cover any suitable pallet load build structure RPAL heightPH (see FIG. 3H) such as, for example, heights of 60 inches, 70 inchesand 80 inches; while in other aspects the field(s) of view FOV1-FOV4 maycover heights less than 60 inches or more than 80 inches.

In one aspect, each of the camera(s) 310C1-310C4 may have a 176 pixel X132 pixel resolution; while in other aspects each, or one or more, ofthe camera(s) 310C1-310C4 may have a higher resolution (e.g. a 320 pixelX 240 pixel resolution or higher), as desired to provide a desiredminimum depth map defining about 0.5 inches at the outermost bounds ofthe pallet build three-dimensional space 3DS (so that the depth mapdefinition throughout the captured image of the whole, or predeterminedpart, of the pallet support / pallet build is not less than about 0.5inches). As such, a sufficient resolution is provided by the visionsystem 300 to resolve lattice features of the pallet support SPAL todefinition so that planarity across the pallet is determined and fullyestablished for placing a stable first layer PL1 of case units CU on thepallet support SPAL as will be described herein. Sufficient resolutionmay also be provided to resolve case unit features (e.g., such as caseedges) so that planarity across a top of each layer PL1-PL4 (see FIG.3H) is determined and fully established for placing a stable layerPL2-PL5 on top of a previously placed layer PL1-PL4. The resolution ofthe camera(s) 310C1-310C4 may be such that minimal processing isrequired to resolve the case unit features (e.g. case unit edges) suchthat the case unit features are resolved in real time substantially fromthe images as received by the controller 10C.

Referring now to FIGS. 3G, 4, 4A and 4B, in one aspect, the controller10C is configured so as to determine, in real time, from thecorresponding real time three-dimensional imaging data, a pallet supportvariance PSV (e.g. a quality of the pallet support SPAL) and/or anarticle unit variance AUV (e.g. a variance in placement of the case unitCU from a planned placement of the pallet load article unit CU as willbe described herein) of at least one of the pallet load article units CUin the pallet load build BPAL with respect to a predetermined reference.The controller 10C is also configured to generate, in real time, anarticulated robot motion signal 390 dependent on at least one of thereal time determined pallet support variance PSV or article unitvariance AUV, where the articulated robot motion signal 390 is generatedin real time so as to be performed real time by the at least onearticulated robot 14 between placement, by the at least one articulatedrobot 14, of at least one pallet load article unit CU and a seriallyconsecutive pallet load article unit CU enabling substantiallycontinuous building of the pallet load build BPAL. In one aspect, the atleast one articulated robot motion signal 390 generated by thecontroller 10C is a stop motion signal along a pick/place path 399 ofthe at least one articulated robot 14, a slow motion signal along thepick/place path 399 of the at least one articulated robot 14, or a moveto a safe position along safe stop path 398 of the at least onearticulated robot 14, where the safe stop path 398 is different from thepick/place path 399. In one aspect, the articulated robot motion signal390 generated by the controller 10C is a place position signal setting aplace position of at least another pallet load article unit CU based onthe pallet support variance or the article unit variance AUV.

The controller 10C is configured so as to determine, in real time, fromthe corresponding real time three-dimensional imaging data, the palletsupport variance PSV, where for example, the vision system 300 imagesthe pallet support SPAL disposed on the pallet building base 301 toobtain a three-dimensional image of the pallet support SPAL withsufficient definition to discern the lattice features 410 of the palletsupport SPAL as described above. Here the pallet support variance PSVmay be one or more of unevenly spaced lattice features 410 (e.g., spacesbetween lattice features forming peak/valleys in a case unit seatsurface - FIG. 4 ), missing portions 400 of lattice features 410 (e.g.,missing board surface(s) - FIGS. 4A and 4B), height differences (e.g.protrusions and/or depressions) or any other defect in the palletsupport SPAL that may affect the stability of the case units CU placedupon the pallet support SPAL (e.g. pallet planarity). In one aspect, thecontroller 10C is configured to reject the pallet support SPAL if thepallet support variance PSV exceeds thresholds from a predeterminedreference (and send stop bot signal until replaced). For example, if themissing portion 400 of the lattice features 410 are greater than apredetermined area or if the spacing between lattice features 410 isgreater than a predetermined distance, the pallet support SPAL isrejected and case units will not be placed until the defective palletsupport SPAL is replaced. If the pallet support SPAL is within thepredetermined thresholds, the controller 10C is configured to resolve acase unit seating surface planar variance (e.g., as noted above, missingboard surface, protrusions, depressions, lattice spacing formingpeak/valleys in the case unit seat surface) relative to a pose of thebase layer PL1 of case units CU (e.g., the positions of each case unitin the three-dimensional space X, Y, Z, θ, α, µ) and confirm or modify(compensates) planned case pose based on the article unit variance AUV(e.g., determine ΔX, ΔY, ΔZ, Δθ, Δα, Δµ) to case plan (X, Y, Z) and/orbased on the pallet support variance PSV (i.e., seat pitch, board edge,etc.) for adaptive pose with higher resultant case stability. Thecontroller may also identify a reduced robot 14 movement speed or modifya robot 14 place trajectory 399 (FIG. 3 ) to generate a desired caseunit CU place pose (e.g., position in the three-dimensional space X, Y,Z, θ, α, µ).

In one aspect, the controller 10C is configured to set a pallet supportbase datum DTM (FIG. 3H) of the pallet support SPAL, imaged by the atleast one three-dimensional camera 310C, from the pallet supportvariance PSV, which pallet support base datum DTM resolves local basesurface variance at each different article unit place location LC1- LCn(see FIG. 2 ) on the pallet support SPAL and defines a real time localarticle unit position base reference for articulated robot 14 placementof the at least one article unit CU of a base article unit layer PL1 ofthe pallet load build BPAL. In one aspect, the pallet support base datumDTM defines base planarity of the pallet support SPAL, and thecontroller 10C is configured to send a signal (such as the usercooperation signal 391 - FIG. 3 ) to a user of the palletizer cell 10 orother operator of the warehouse system 100WS, with informationdescribing a base planarity (FIG. 3H) characteristic (e.g. missing boardsurfaces, protrusions, depressions, lattice spacing forming peak/valleysin the case unit seat surface), to enable selection of the at least onepallet load article unit CU of the base layer PL1, from a number ofdifferent size pallet load article units CU of the pallet load PAL, andof a corresponding placement location LC1-LCn on the pallet support SPALso as to form the base layer PL1 based on the base planarity (FIG. 3H).

In one aspect, the base planarity characteristic information describesplanarity variance for a corresponding area (such as one of placementlocations LC1-LCn) of the base datum DTM in real time, and thecontroller 10C is configured to identify, from the different size palletload article units CU of the pallet load PAL, one or more pallet loadarticle units CU sized so as to seat stably on the corresponding area soas to form the base layer PL1. In one aspect, the pallet support basedatum DTM defines a base planarity of the pallet support, and thecontroller 10C is configured to select the at least one pallet loadarticle unit CU of the base layer PL1, from a number of different sizepallet load article units CU of the pallet load PAL, and a correspondingplacement location LC1-LCn on the pallet support SPAL so as to form thebase layer PL1 based on the base planarity. In one aspect, thecontroller 10C is configured so as to determine in real time, from thereal time three-dimensional imaging data (such as the images shown inFIG. 4-4B of the pallet support SPAL and any variances on the palletsupport SPAL captured by the vision system 310) and substantiallycoincident with setting of the pallet support base datum DTM, lateralbounds ER (see FIGS. 3H-3J) of the pallet support base datum DTM,wherein at least one of the lateral bounds ER forms a lateral referencedatum LTDM (FIG. 3J) defining lateral position and orientation of thepallet load build BPAL on the pallet load base datum DTM, and forming areference frame FER (FIG. 3J) for placement position of the at least onepallet load article unit CU with the at least one articulated robot 14building the pallet load build BPAL.

In one aspect, referring also to FIG. 4C, the predetermined referenceincludes a predetermined pallet support inspection reference IR defininga predetermined pallet support structure reference characteristic SR(e.g. such as a planarity PLN or non-defective support surface NDSS of apallet TPAL -FIG. 4C). In one aspect, the pallet support variance PSV isa difference determined by the controller 10C between the predeterminedpallet support structure reference characteristic SR and acharacteristic of the pallet support SPAL (e.g., such as the missingboard surfaces, protrusions, depressions, lattice spacing formingpeak/valleys in the case unit seat surface described above), disposed onthe building base 301 and imaged by the at least one three-dimensionalcamera 310C, corresponding thereto and resolved in real time by thecontroller 10C from the three-dimensional imaging data (such as theimages illustrated in FIG. 4-4B). In one aspect, the controller 10C isconfigured to compare the determined pallet support variance PSV withthe predetermined threshold (as described above) for at least onepredetermined pallet support structure reference characteristic SR, andgenerate an articulated robot motion signal 390 (commanding articulatedrobot stop and/or changing a articulated robot motion path and/ortrajectory) if the determined pallet support variance PSV is greaterthan the predetermined threshold, and if the determined pallet supportvariance PSV is less than the predetermined threshold, generate anarticulated motion signal 390 that embodies an article unit CU placeposition signal identifying placement of at least another pallet loadarticle unit CU building the pallet load build BPAL to the at least onearticulated robot 14. In one aspect, the predetermined referenceincludes a predetermined reference position of the at least one palletload article unit CU in a predetermined reference pallet load build BPALcorresponding to the building pallet load build on the pallet supportSPAL.

As also noted above, and still referring to FIGS. 3G, 4, 4A and 4B, thecontroller 10C is configured to identify (e.g., case unit position isvalidated to a pallet build plan) an article unit variance AUV (e.g. avariance in placement of the case unit CU from a planned placement ofthe pallet load article unit CU as described herein) of at least one ofthe pallet load article units CU in the pallet load build BPAL withrespect to a predetermined reference (where the reference may be apredetermined place case pose specified by the pallet build plan). Inone aspect, the article unit variance AUV is a difference determined bythe controller 10C between a position, resolved in real time by thecontroller 10C from the three-dimensional imaging data, of the at leastone pallet load article unit CU in the pallet load build BPAL and thepredetermined reference position (as determined by the planned placementof the pallet load article unit CU) of the at least one pallet loadarticle unit CU. For example, the controller 10C is configured todetermine if case unit placement is within a predetermined threshold foroverhang e_(oi) (e.g. relative to supporting edges - see FIG. 3H for anegative overhang where the superior case unit is inward of the edge EDand see FIG. 3K for a positive overhang where the superior case unitextends outward of the edge ED); is within a predetermined eccentricitye_(CLi) (see FIG. 3H) with respect to stack centerline CL (e.g., such asa centerline CL of stack S1 in FIG. 2H) to determine a stability ofstack; and is within a seat flatness and height for superior structurescases/layers (such as case units within the superior layer PL2-PL5. Thecontroller 10C is configured to resolve such article unit variances AUVrelative to pose of superior seated and/or adjacent case place poses andmodify case unit CU placement to compensate for superior or adjacentplace poses and/or compensate the robot motion in a manner similar tothat described above so that the stacks S1-Sn of case units CU (see FIG.2 ) are stable. The controller is configured to stop robot motion if apose of a case unit placed on or being placed on the pallet load buildBPAL exceeds the thresholds and generates a deviant condition. In oneaspect, any suitable user cooperation signal 391 of such deviantcondition may be sent (e.g., aurally and/or visually) to a user operatoridentifying the deviant condition and location of the deviant case unitCU. The user cooperation signal 391 may include a collaborating action(e.g., such as repositioning the deviant case unit CU by hand) withinthe robot cycle time from case placement and a next sequential caseplacement, where for example, the controller 10C stops movement of therobot 14, slows movement of the robot 14, and/or moves the robot 14 to asafe area along safe stop path 398.

In one aspect, the controller 10C is configured so as to determine, inreal time, from the corresponding real time three-dimensional imagingdata, a build pallet load variance BPV (FIG. 3 ) with respect to apredetermined reference. The build pallet load variance BPV includesidentification of at least one of a presence of an extraneous object 233(see FIG. 2 where the extraneous object is illustrated as a tool but inother aspects the extraneous object may be a fallen or misplaced caseunit, part of the robot 14 or any other object that does not belong inthe detected position) in the pallet load build BPAL and of amispresence (i.e., a mispresence is an erroneous position, orientationor missing presence of the article unit CU as illustrated in FIG. 5-10D)of at least one pallet load article unit CU from the pallet load buildBPAL. The controller 10C is also configured so as to generate in realtime an articulated robot motion signal 390 dependent on the real timedetermined build pallet load variance BPV, and the articulated robotmotion signal 390 being generated real time so as to be performed realtime by the articulated robot 14 substantially continuously building thepallet load build BPAL substantially coincident with imaging of thepallet load build BPAL, between placement, by the articulated robot 14,of serially consecutive pallet load article units CU, placed immediatelyprior and immediately after imaging of the pallet load build BPALshowing the determined build pallet load variance BPV. In still anotheraspect, the controller 10C is configured so as to generate in real timea robot motion signal 390 and a user (e.g. an operator of the palletizercell 10 or other personnel of the warehouse 100WS) cooperation signal391, both dependent on at least one of the real time determined buildpallet load variance BPV, wherein the user cooperation signal 391defines to the user a deviant condition of the pallet load build BPALand a cooperative action of the user so as to resolve the deviantcondition depending on the determined at least one extraneous presenceand mispresence. It should be understood that the controller 10C may beconfigured to determine one or more of the pallet support variance PSV,the article unit variance AUV and the build pallet load variance BPV.

In accordance with the aspects of the disclosed embodiment, the visionsystem 310 imaging and responsive (feedback) input (e.g. feedback loopBFL), to the robot(s) 14 is decoupled/independent from robot motion,hence enabling a substantially continuous and adaptive pallet load buildas described herein. The vision system 310 is capable of detecting andresolving (alone or in combination with controller 10C) within thepallet load build BPAL one or more of a quality of the pallet supportSPAL and an identification (validated to plan) of placed case posevariances from reference (where the reference may be a predeterminedplace case pose specified by a pallet build plan).

In one aspect, referring again to FIGS. 3 and 3A-3N, the vision system300 is fixed in position relative to the frame 300F, and as such isindependent of robot 14 movement/position. For example, each camera310C1-310C4 of the at least one camera 310C is positioned on the frameso that a respective field of view FOV1-FOV4 images at least apredetermined part of the pallet load build BPAL (e.g. pallet supportSPAL, at least part of the pallet load build structure RPAL, etc.) in athree-dimensional space 3DS, as described herein. As can be seen in thefigures, in one aspect, the fields of view FOV1-FOV4 may overlap; whilein other aspects the fields of view FOV1-FOV4 may not overlap. Where thefields of view FOV1-FOV4 overlap, the images from each camera may becombined to form an uninterrupted substantially continuous image of thewhole, or at least a predetermined part, of the pallet load buildstructure RPAL within the three-dimensional space 3DS (e.g., where thecombined image has a resolution as described herein). Each of thecameras 310C1-310C4 of the at least one camera 310C is calibrated toregister each three-dimensional image (e.g. a two-dimensional image anda depth map point cloud (see FIGS. 5-10D for exemplary images includingthe depth map point cloud), where the image and depth map point cloudare created with respect to a global reference frame (see FIG. 3G andreference frame X, Y, Z, θ, α, µ) of the three dimensional space 3DS) toa common reference point, so as to synchronize each camera 310C1-310C4with respect to the other cameras 310C1, 310C4 and identify overlappedareas of the three-dimensional image.

Each of the cameras 310C1-310C4 are also configured to register thethree-dimensional point clouds corresponding to each robot 14 toidentify pallet load build structure RPAL occlusion zones OZ (Se FIG.5A) formed by the robots 14 extending into the three-dimensional space3DS. For example, each robot 14 is dynamic (e.g. has an extend motion, aretract motion and a static place/pick motion) within thethree-dimensional space 3DS. The vision system 310 alone, or incombination with the controller 10C is configured, with any suitablealgorithms, to remove the occlusion zones OZ formed by the robot 14 sothat the vision system provides a substantially unobstructed view of thepallet load build structure RPAL / pallet load build BPAL. For example,the occlusion zones OZ are registered by the respective cameras 310C1,310C4 with the controller 10C. The controller 10C is configured tocompare the three-dimensional image data (e.g. current image data and/orpast image data of the pallet load build BPAL / pallet load buildstructure RPAL) with the occlusion zones OZ and subtract the occlusionzones OZ from (or otherwise ignores the occlusion zone OZ in) thethree-dimensional image so that images of the case units CU (see caseunits 1-5 in FIG. 5A) are substantially unobstructed. In one aspect, theat least one camera 310C is communicably coupled to the controller 10Cso the controller 10C registers (in any suitable register such asregister 10CR - FIG. 3 ), from the at least one camera 310C, real timethree-dimensional imaging data embodying different correspondingthree-dimensional images of each different one of the pallet loadarticle units, of the pallet load build BPAL.

In one aspect, referring to FIG. 3B, the cameras 310C1-310C4 of thevision system 310 are configured so that the cameras 310C1-310C4 areoperated and images are obtained/captured by the cameras 310C1-310C4 ina cascading (or sequential) manner. For example, a trigger TG (such asfeedback from the robot 14 to the controller 10C that a case unit CU waspicked for placement by or placed by the robot 14 - e.g., the triggermay be based on robot position or any other suitable criteria) may bereceived by the controller 10C, which trigger TG causes the controller10C to send an imaging command to the vision system 310. The cameras310C1-310C4 may be coupled to each other with any suitable input/outputinterlock LCK so that the cameras 310C1-310C4 sequentially capturerespective images of the pallet load build BPAL / pallet build structureRPAL (e.g. camera 310C1 captures a respective image, then camera 310C2captures a respective image, then camera 310C3 captures a respectiveimage and so on, until all cameras in the vision system 300 capturetheir respective images). It should be understood that while thesequential imaging is described as being captured in the order of camera310C1, camera 310C2, camera 310C3, camera 310C4, in other aspects thecameras 310C1-310C4 may capture their respective images in any suitableorder. Here, each camera 310C1-310C4 of the vision system 310 acquires arespective image and depth map independent of the other cameras310C1-310C4 (e.g. to avoid interference between cameras). In one aspect,the cameras 310C1-310C4 may be hardwired serially to each other so thateach camera input/output interlock LCK triggers imaging by a subsequentcameral in the sequence of cameras. For example, the camera 310C1 mayreceive the imaging signal from the controller 10C, the camera 310C2 mayreceive the imaging signal from camera 310C1, the camera 3103 mayreceive the imaging signal from camera 310C2 and so on. As may berealized, the imaging sequence of the cameras 310C1-310C4 may occurwithin a predetermined time period so that sufficient time is providedto the controller 10C, upon receipt of the image and depth map, forprocessing the image and depth map and providing an adaptive response tothe robot(s) 14 (e.g. imaging is completed in about 0.4 seconds).

In one aspect, the controller 10C determines the features of the palletbuild structure RPAL (e.g. edges of the case units CU, features of thepallet support, etc.) directly from the point cloud (defining the imageddepth map) for each camera 310C1-310C4 independently and substantiallycoincidentally. Determination of the features of the pallet buildstructure RPAL directly from the point cloud is effected by simplifyingthe point cloud by resolving robot occlusions OZ (see FIG. 5A), asdescribed above, and other point cloud simplification to removeareas/spaces of no interest (e.g. by subtracting these areas/spaces fromthe image in a manner substantially similar to that described above). Itis noted that the areas/spaces of no interest may include areas outsidethe case unit placement area of the pallet load build structure RPAL.The controller 10C is configured with any suitable algorithms toresolve, from the point cloud, the case unit edges to determine caseunit size and location. The controller 10C is also configured todetermine, from the point cloud, variances for each camera point cloudindependently and substantially coincidentally with respect to thereference feature (as noted herein). The controller 10C is alsoconfigured to resolve any overlapped portions of the images from thecameras 310C1-310C4 to eliminate duplication. The controller 10C isfurther configured to determine the appropriate adaptive response forthe robot(s) as described herein.

Referring to FIGS. 2, 3, 5 and 5A-5G, placed case variancedetection/determination by the controller 10C includes detection of casemispresence/mispose such as detection of missing case and/or detectionof an incorrectly placed case. FIGS. 5, 5A-5G illustrate a sequencethree dimensional point cloud images, obtained from the vision system310, of case unit CU placement on a pallet support SPAL where the caseunit CU8 is placed so as to overlap case unit CU6 (i.e. case unit ismisposed and a signal 390, 391 are generated as described herein toeffect repositioning the misposed case unit CU6). It is noted that uponrobot 14 case placement of each case unit CU building the pallet loadbuild structure RPAL / pallet load build BPAL, the vision system 310three-dimensionally images the pallet load build BPAL, wholly or apredetermined portion thereof, for each robot 14 motion indicated as acase place building the pallet load build BPAL, so that the visionsystem 310 serially images the whole (or predetermined portion of)build, to show incremental differences from each case placement toanother subsequent case placement, in effect imaging each case unit CUplaced building the pallet load PAL, facilitating validation (in realtime) of pose and adaptive response as described above.

With respect to the “missing” case detection, imaging the whole (orpredetermined portion of) build in the serial images (see FIGS. 6, 6A-6Hwhere case unit 9 falls from the pallet build and is missing; see alsoFIGS. 7, 7A-7F where case unit 63 falls from the pallet build and ismissing) of the pallet load build BPAL, enables detection of the“missing” case anywhere, at any time (from any occurrence) such asimmediately upon robot place motion from misplacement/non-placement ofthe “missing” case by the robot 14 or from delayed case fall (see FIG.6-6H which show a sequence of images illustrating case unit 9 goingmissing and falling to the floor; and FIG. 7-7F which show a sequence ofimages illustrating case unit 63 going missing and falling to the floor)caused by, e.g., environmental history changes over time in the palletload build PBAL (due to, e.g., deformations from surface loads on casesin the build structure and/or shifting of cases from instability ordynamic impulse, contact to the pallet load build structure RPAL). Realtime detection of the missing case deviant condition (e.g. missing case)causes the controller 10C to send a robot motion signal 390 (similar tothe stop, slow, safe motions described herein) and generate a usercooperation signal 391 with information identifying condition types(e.g., missing, misposed), location, and collaboration action (e.g.,replace/reposition the missing case). Similar corrective action may betaken by the controller 10C for the overlapped/misposed case unit CU6 ofFIG. 5-5G as noted above.

The controller 10C is programmed with the place sequence and case unitlocations (according to the reference plan of the pallet load buildBPAL) for each case CU, and registers the identification of each casebeing placed by the robot 14 (in each given robot pick/place cycle) tothe corresponding place cycle and the corresponding place pose imagedupon placement by the corresponding place cycle. Accordingly thecontroller 10C, from the real time missing case determinations,discriminates the missing case (e.g. in the example shown in FIG. 6-6Hthe missing case is case 6) as being effected by robot 14 misplacement(e.g., the identified missing case is the same case as placed by bot inlast preceding place cycle) or is a missing case due to environmentalhistory effects. As described above, the controller 10C is programmed togenerate different robot 14 and user signals for missing case units CUbased on the discriminated missing case type. For example, if themissing case is due to, for example, bot placement, the controller 10Cmay modify the robot 14 speed or trajectory to place other cases withsimilar proportions and similar poses, and provide a preemptive usercooperation signal 391 in advance of the robot 14 place cycles withsimilar cases/totes.

Similarly the controller 10C is configured to determine and discriminatecase pose variances due to robot 14 place effects (as shown in, e.g.,FIG. 5-5G) and those due to environmental history changes (as shown in,e.g., FIG. 6-7F). Here the controller 10C determines corresponding caseunit CU position compensation based on and as appropriate for each casetype and generates robot motion signals 390 and user cooperation signals391 corresponding to compensation and user response as described herein.As noted herein, each three-dimensional imaging, upon placement of eachgiven case unit CU, images the given placed case unit CU (withregistered identification, corresponding robot place motion andpose/reference place pose) and the whole (or predetermined part) of thepallet load build (from initial case placement to the given placed caseassociated with imaging) as affected by environmental history changes.Accordingly, pose validation/variance amount (as described above) isdetermined real time upon placement of the given case unit CU, for thegiven case unit CU, and for each other case unit CU visible with in theimaging system field of view (e.g., the combined field of view of the atleast one camera 310C). Variances in the pose of the given case unit CUdetermined in real time, based on the imaging upon placement of the caseunit CU, are substantially representative of place effects from robot 14motion.

Variances for each of the other case units CU in the whole (orpredetermined part of) pallet load build BPAL imaged upon placement ofthe given case unit CU, determined in real time, are effected byenvironmental history changes. Thus, the controller 10C may discriminate(from the real time determination of variance and the correlation ofcases and robot 14 place cycles) such variances due to robot 14 placeeffects from such variance due to environmental history changes (e.g.,discrimination of variances by variance cause type). Variancecompensation may be different due to the cause type. As may be realized,variance compensation includes static compensation (also referred toherein as pose compensations) and dynamic compensations (e.g., that aimto compensate for robot 14 dynamics in placing cases). Pose compensation(e.g., determination of Δx, Δy, Δz relative to planned placement) forplacement of subsequent or superposed case unit(s) CU based on (andaccounting for) the determined pose variance of the three-dimensionalimaged case unit CU is substantially homologous for variances effectedby robot 14 placement of the given case CU and for variances effected byenvironmental historic changes in pallet build structure RPAL.

The controller 10C, thus determines the pose variance V_((Δx,) _(Δy,)_(Δz)i) of a three-dimensionally imaged given case CU, (on placement)and of each other case unit CU in the three-dimensionally imaged whole(or predetermined part) of the pallet load build BPAL and with asuitable algorithm determines the pose compensation C_((Δx,) _(Δy,)_(Δz)i) for the subsequent, superior, or superposed case unit(s) CU tobe placed freely according to the place sequence anywhere in the palletload build structure RPAL/pallet load build BPAL, and the posecompensation’s determination is effected as described herein in realtime within the robot(s) 14 place cycle motion, and signaled to therobot(s) 14 to be performed in the next place cycle motion ifappropriate. With each imaging, the controller 10C may further updatethe pose validity/determined variance for each of the other case unitsCU in the three-dimensionally imaged whole (or predetermined part) ofthe pallet load PAL from the preceding determination, and may furthercorrelate the location or proximity of each case unit CU and the placedgiven case location. The variance updates, or changes in pose varianceas identified from the updates may be further analyzed, by thecontroller 10C for identification of possible trends in variances, andpallet load build BPAL stability that may undermine load build stability(the trends may be resolved automatically by the controller 10C or withuser assistance).

Similar to the missing case determination, discrimination (e.g., by thecontroller 10C) as to cause of the pose variance type, enablesdetermination by the controller 10C of dynamic compensation (e.g., robot14 speed, trajectory, etc.) desired to place subsequent superior and/orsuperposed case units CU, freely anywhere in the pallet load PALaccording to the place sequence and independent of prior case CUplacement. In effect, the controller 10C correlates the pose variance V_((Δx, Δy, Δz)i) of the imaged given placed case unit CU (i.e., thevariance substantially due to robot place motion) to bot case endeffector kinematics just prior to and at the case place position anddetermines appropriate changes in bot kinematics for subsequent superiorand/or superposed case units CU placed (e.g., based on the correlationand a suitable algorithm/empiric relationship predicting resultantchanges in case place pose from changes in robot place motionkinematics). The dynamic compensation desired is signaled to the robot14 and user as appropriate and the robot 14 kinematic contribution topose variance, may thus be discriminated and via compensation over anumber of rate cycle, be tuned out, resulting in more repeatable caseplacement pose (e.g., minimizing variances from robot 14 kinematics).

As may be realized, determination of and application of robot 14 dynamiccompensation is not limited to occasions of variances substantially dueto robot 14 place motion, and may also be determined from the posevariance proximity to thresholds (e.g., stack/pallet overhangs, caseunit stack eccentricity, etc., arising from either bot placement of thegiven place case, and/or environmental historic changes) and likelybased on predetermined criteria, to be adversely affected resulting inunacceptable pose from subsequent superior or superposed case unit CUplacement. Case units CU with such pose variance may be consideredlimited stability or limited positioned case units, and resultant robot14 kinematic compensation may be determined (e.g., in a manner similarto that described herein) in real time, so that subsequent, superior orsuperposed case units CU may be placed freely anywhere on the palletload PAL, according to the predetermined place sequence and independentof immediate prior case unit CU placement. The dynamic compensationdesired is signaled to the robot 14 and user as appropriate (e.g., therobot 14 is signaled so that the robot kinematic deceleration trajectoryis slowed down in respective motion cycle, and the user is signaled tomonitor robot 14 place on corresponding robot 14 place motion cycle).

Determination in real time (enabled and facilitated bythree-dimensionally imaging the pallet build load structure RPAL, withthe vision system 310, upon placement of each given case unit CU placebuilding the pallet build load structure RPAL) of case pose variance andmissing case(s) are examples of the broader capabilities of the palletbuilding apparatus described herein to determine in real timemispresence of cases CU and/or other extraneous or extraordinary (e.g.,with respect to pallet build load structure RPAL) articles and objects,causing deviant conditions of the pallet build (e.g. such as case unitsCU falling onto the pallet as illustrated in the sequence of imagesprovided in FIG. 9-9E, where case unit 4 falls onto the pallet and FIG.10-10D, where case unit 6 is dropped by the robot 14 onto the pallet inan incorrect position) and providing in real time motion commandssignals to the robot 14 (e.g., including stopping the robot 14 on acurrent path and kinematic trajectory, changing the robot 14 kinematictrajectory, slowing the robot 14 on the current path, changing a path ofthe robot 14 for example to a safe path (towards a zone in which robotpath to stop will not encounter potential obstruction, andslowing/stopping robot 14 motion in a safe position)). As noted, thecontroller 10C registers/catalogues each case unit(s) and its placementin a case unit register 10CR (and updates such register 10CR) with eachgiven case unit CU placement in the pallet load build structure RPAL. Inone aspect, the controller 10C is configured so as to determine, in realtime, from the corresponding real time three-dimensional imaging data, abuild pallet load variance BPV with respect to the predeterminedreference (described above), the build pallet load variance BPV beingdeterminative of at least one of an extraneous presence, of anextraneous object in the pallet load build BPAL, and of a mispresence ofat least one article unit CU from the pallet load build BPAL.

Three-dimensional imaging of the whole (or predetermined part) of thepallet load build structure RPAL, images the extraneous presence of anextraneous object (e.g., a case unit CU, including a fallen case (suchas described above), from somewhere in the pallet load build BPAL, thatmay or may not be a missing case, or from a region nearby the palletload shelf (e.g. the pallet building base 301), a misplaced slip sheet,or any other object, workpiece, hand tool, extraneous robot 14 part,etc.) in static position anywhere in the pallet load build structureRPAL upon imaging. The controller 10C determines the presence of theextraneous object (as noted above) anywhere (e.g., each location ofstructure) from the three-dimensional imaging data in real time (in amanner similar to that described herein), and that the presence isextraneous (e.g., the object is extraneous) by comparing the objectshape resolved from the three-dimensional imaging data, to plan shapes(e.g., is the object shape regular such as conforming toshapes/dimensions of a case unit CU or other pallet load article unit CUidentified by the pallet load plan) for conformance.

The controller 10C is configured so as to generate in real time a robotmotion signal 390 and a user cooperation signal 391, both dependent onat least one of the real time determined build pallet load variance BPV,the robot motion signal 390 being generated real time so as to beperformed real time by the robot(s) 14 substantially continuouslybuilding the pallet load build BPAL substantially coincident withimaging of the pallet load build BPAL, between placement, by therobot(s) 14, of serially consecutive pallet load article units CU,placed immediately prior and immediately after imaging of the palletload build BPAL showing the determined build pallet load variance BPV.For example, if the shape is a non-conformal shape, the controller 10Cidentifies if the position of the non-conformal shape is interferingwith (e.g., a deviant condition) the next (or next series) of placeactions of the substantially continuous place motion cycle, and if socommands the robot 14 real time with corresponding robot motion signal390 (as described above). The controller 10C may also signal the deviantcondition to the user, noting the extraneous presence, location, androbot 14 safe timing/condition so that the user may remove or repositionthe non-conformal object corresponding to the non-conformal shape in theimage. The controller 10C is configured, in the event the shape resolvedis generally regular (e.g., straight or other geometrically definedfeatures/shapes) to resolve attitude or orientation with respect to therelevant frame of reference to determine if the object is a tilted case(as illustrated in FIG. 5-5G) (this presents a further variant of thepose determination) and in response identify as such to the user andmotion command the robot 14. If the shape is regular, the controller 10Cmay compare the place registry and missing case unit(s) and determine ifthe object is a fallen case and address tilted cases similarly.

Referring now to FIGS. 1, 2, 3 and 11 , an exemplary operation of theadaptive palletizer system 300 will be described. In one aspect, anysuitable controller of the warehouse system 100WS (such as controller10C of the adaptive palletizer system 300) defines and sets a referencepallet load build structure REFPAL (FIG. 2 ) (FIG. 11 , Block 1100). Thereference pallet load structure REFPAL may be defined and set from anorder and/or load structure sequence output from the automated storageand retrieval system 100. The ordered case units CU are identified andregistered in any suitable manner and correlated to reference robot 14place move cycle sequences building the pallet load build structure RPAL(FIG. 11 , Block 1110). For example, ordered case units may beidentified by any suitable controller, such as controller 199C forpicking from the storage and retrieval system 100 storage structure 130.These identified case units CU may be communicated from controller 199Cto controller 10C of the adaptive palletizer system 300 so that thecontroller 10C correlates the identified case units CU to robot(s) 14move sequences for building a stable pallet load build structure RPAL.It is noted, as described above, that each palletizer cell 10 mayinclude a corresponding case unit inspection cell 142 (FIG. 1 ) forverifying and registering the received ordered case units CU in theregister 10CR.

The vision system 310 three-dimensionally images the pallet support SPALand determined pallet variances (as described above) and generates anynecessary user cooperation signals 391 and/or robot motion signals 390in response to the determined pallet variances (FIG. 11 , Block 1120).The controller 10C is configured to command the robot(s) 14 for placingthe case unit(s) CU on the pallet support SPAL for building the palletload build structure RPAL. For example, for each case(s), the case(s) isplaced with a bot place cycle to build the pallet load build structureRPAL. The controller 10C is configured to register each case placement(e.g. in register 10CR) for a corresponding reference place move and areference predetermined location (as determined from the referencepallet load build structure REFPAL) (FIG. 11 , Block 1130). Uponplacement of each case unit CU in the pallet load build structure RPAL,the vision system 300 captures (as described above) a three-dimensional,time of flight, image of the whole, or predetermined portion, of the asbuild pallet load structure (e.g. the pallet load build BPAL on thepallet support SPAL), where the image is inclusive of previously placedcase unit(s) CU (FIG. 11 , Block 1140). The controller 10C isconfigured, as described above, to determine in real time from thethree-dimensional image data variances (which include, as describedabove, mispresence, extraneous presence, mispose); and generate realtime compensatory bot motion signals 390 and compensatory usercooperation signals 391, as described above (FIG. 11 , Block 1150).

Referring now to FIGS. 1, 2, 3 and 12 , an exemplary operation of theadaptive palletizer system 300 will be described. In one aspect, apallet load is automatically built from pallet load article units CUonto a pallet support SPAL by defining, with a frame 300F, a palletbuilding base 301 for the pallet support SPAL (FIG. 12 , Block 1200). Atleast one articulated robot 14 connected to the frame 300F transportsand places (as described above) the pallet load article units CUserially onto the pallet support SPAL so as to build the pallet load PALon the pallet building base 301. (FIG. 12 , Block 1210). The articulatedrobot 14 motion is controlled, relative to the pallet building base 301,with the controller 10C, which is operably connected to the at least onearticulated robot 14, to effect therewith a pallet load build BPAL ofthe pallet load PAL (FIG. 12 , Block 1220). Three-dimensional imaging ofthe pallet support SPAL on the pallet building base 301 and of thepallet load build BPAL on the pallet support SPAL is generated (asdescribed above) with at least one three-dimensional, time of flight,camera 310C, where the at least one three-dimensional camera 310C iscommunicably coupled to the controller 10C (FIG. 12 , Block 1230). Thecontroller 10C registers, from the at least one three-dimensional camera310C, real time three-dimensional imaging data embodying differentcorresponding three-dimensional images of the pallet support SPAL and ofeach different one of the pallet load article units CU, and of thepallet load build BPAL (FIG. 12 , Block 1240).

In one aspect, the controller 10C determines, in real time, from thecorresponding real time three-dimensional imaging data, a pallet supportvariance PSV (FIG. 3 ) or article unit variance AUV (FIG. 3 ) of atleast one of the pallet load article units CU in the pallet load buildBPAL with respect to the predetermined reference (as described above).The controller 10C also generates in real time an articulated robotmotion signal 390 dependent on at least one of the real time determinedpallet support variance PSV or article unit variance AUV, thearticulated robot motion signal 390 being generated real time so as tobe performed real time by the at least one articulated robot 14 betweenplacement, by the at least one articulated robot 14, of at least onepallet load article unit CU and a serially consecutive pallet loadarticle unit CU enabling substantially continuous building of the palletload build BPAL (FIG. 12 , Block 1250).

In one aspect, the controller 10C determines in real time, from thecorresponding real time three-dimensional imaging data, a build palletload variance BPV with respect to the predetermined reference (asdescribed above) (FIG. 12 , Block 1260). The build pallet load varianceBPV includes identification of at least one of a presence of anextraneous object 233 in the pallet load build BPAL and a mispresence ofat least one pallet load article unit CU from the pallet load buildBPAL. The controller 10C generates, in real time an articulated robotmotion signal 390 (FIG. 12 , Block 1270), or an articulated robot motionsignal 390 and a user cooperation signal 391 (FIG. 12 , Block 1280),dependent on at least one of the real time determined build pallet loadvariance BPV, where the articulated robot motion signal 390 is generatedreal time so as to be performed real time by the articulated robot 14substantially continuously building the pallet load build BPALsubstantially coincident with imaging of the pallet load build BPAL,between placement, by the articulated robot 14, of serially consecutivepallet load article units CU, placed immediately prior and immediatelyafter imaging of the pallet load build BPAL showing the determined buildpallet load variance BPV.

In accordance with one or more aspects of the disclosed embodiment apallet building apparatus for automatically building a pallet load ofpallet load article units onto a pallet support is provided. The palletbuilding apparatus comprises:

-   a frame defining a pallet building base for the pallet support;-   at least one articulated robot connected to the frame and configured    so as to transport and place the pallet load article units serially    onto the pallet support so as to build the pallet load on the pallet    building base;-   a controller operably connected to the at least one articulated    robot and configured to control articulated robot motion, relative    to the pallet building base, and effect therewith a pallet load    build of the pallet load;-   at least one three-dimensional, time of flight, camera disposed so    as to generate three-dimensional imaging of the pallet support on    the pallet building base and of the pallet load build on the pallet    support;-   wherein the at least one three-dimensional camera is communicably    coupled to the controller so the controller registers, from the at    least one three-dimensional camera, real time three-dimensional    imaging data embodying different corresponding three-dimensional    images of the pallet support and of each different one of the pallet    load article units, and of the (building) pallet load build, and-   the controller is configured so as to determine, in real time, from    the corresponding real time three-dimensional imaging data, a pallet    support variance or article unit variance of at least one of the    pallet load article units in the pallet load build with respect to a    predetermined reference, and generate in real time an articulated    robot motion signal dependent on at least one of the real time    determined pallet support variance or article unit variance, the    articulated robot motion signal being generated real time so as to    be performed real time by the at least one articulated robot between    placement, by the at least one articulated robot, of at least one    pallet load article unit and a serially consecutive pallet load    article unit enabling substantially continuous building of the    pallet load build.

In accordance with one or more aspects of the disclosed embodiment theat least one three-dimensional camera is configured so as to effectthree-dimensional imaging of the pallet support on the pallet buildingbase and of the pallet load build on the pallet support with the atleast one articulated robot effecting substantially continuouspick/place cycles from an input station and placing each of the palletload article units building the pallet load on the pallet building base.

In accordance with one or more aspects of the disclosed embodiment theat least one three-dimensional camera is configured so as to effectthree-dimensional imaging of each respective pallet load article unitsubstantially coincident with placement of the respective pallet loadarticle unit by the at least one articulated robot effectingsubstantially continuous pick/place cycles from an input station andplacing the pallet load article unit building the pallet load buildsubstantially continuously.

In accordance with one or more aspects of the disclosed embodiment theat least one articulated robot motion signal generated by the controlleris a stop motion signal along a pick/place path of the at least onearticulated robot, a slow motion signal along the pick/place path of theat least one articulated robot, or a move to a safe position along safestop path of the at least one articulated robot, different from thepick/place path.

In accordance with one or more aspects of the disclosed embodiment thearticulated robot motion signal generated by the controller is a placeposition signal setting a place position of at least another pallet loadarticle unit based on the pallet support variance or the article unitvariance.

In accordance with one or more aspects of the disclosed embodiment thepredetermined reference includes a predetermined pallet supportinspection reference defining a predetermined pallet support structurereference characteristic.

In accordance with one or more aspects of the disclosed embodiment thedetermined pallet support variance is a difference determined by thecontroller between the predetermined pallet support structure referencecharacteristic and a characteristic of the pallet support, imaged by theat least one three-dimensional camera, corresponding thereto resolved inreal time by the controller from the three-dimensional imaging data.

In accordance with one or more aspects of the disclosed embodiment thecontroller is configured to compare the determined pallet supportvariance with a predetermined threshold for at least one predeterminedpallet support structure reference characteristic, generate anarticulated robot motion signal (commanding articulated robot stopand/or changing a articulated robot motion path and/or trajectory) ifthe determined pallet support variance is greater than the predeterminedthreshold, and if the determined pallet support variance is less thanthe predetermined threshold, generate an article unit place positionsignal identifying placement of at least another pallet load articleunit building the pallet load build to the at least one articulatedrobot.

In accordance with one or more aspects of the disclosed embodiment thecontroller is configured to set a pallet support base datum of thepallet support, imaged by the at least one three-dimensional camera,from the pallet support variance, which pallet support base datumresolves local base surface variance at each different article unitplace location on the pallet support and defines a real time localarticle unit position base reference for articulated robot placement ofthe at least one article unit of a base article unit layer of palletload build.

In accordance with one or more aspects of the disclosed embodiment thepallet support base datum defines base planarity of the pallet support,and the controller is configured to send a signal to a user, withinformation describing base planarity characteristic, to enableselection of the at least one pallet load article unit of the baselayer, from a number of different size pallet load article units of thepallet load, and of a corresponding placement location on the palletsupport so as to form the base layer based on base planarity.

In accordance with one or more aspects of the disclosed embodiment thebase planarity characteristic information describes planarity variancefor a corresponding area of the base datum in real time, and thecontroller is configured to identify, from the different size palletload article units of the pallet load, one or more pallet load articleunits sized so as to seat stably on the corresponding area so as to formthe base layer.

In accordance with one or more aspects of the disclosed embodiment thepallet support base datum defines base planarity of the pallet support,and the controller is configured to select the at least one pallet loadarticle unit of the base layer, from a number of different size palletload article units of the pallet load, and a corresponding placementlocation on the pallet support so as to form the base layer based onbase planarity.

In accordance with one or more aspects of the disclosed embodiment thecontroller is configured so as to determine in real time, from the realtime three-dimensional imaging data and substantially coincident withsetting of the pallet support base datum, lateral bounds of the palletsupport base datum, wherein at least one of the lateral bounds forms alateral reference datum defining lateral position and orientation of thepallet load build on the pallet load base datum, and forming a referenceframe for placement position of the at least one pallet load articleunit with the at least one articulated robot building the pallet loadbuild.

In accordance with one or more aspects of the disclosed embodiment thepredetermined reference includes a predetermined reference position ofthe at least one pallet load article unit in a predetermined referencepallet load build corresponding to the building pallet load build on thepallet support.

In accordance with one or more aspects of the disclosed embodiment thedetermined article unit variance is a difference determined by thecontroller between a position, resolved in real time by the controllerfrom the three-dimensional imaging data, of the at least one pallet loadarticle unit in the pallet load build and the predetermined referenceposition of the at least one pallet load article unit.

In accordance with one or more aspects of the disclosed embodiment apallet building apparatus for automatically building a pallet load ofpallet load article units onto a pallet support is provided. The palletbuilding apparatus comprises:

-   a frame defining a pallet building base for the pallet support;-   at least one articulated robot connected to the frame and configured    so as to transport and place the pallet load article units serially    onto the pallet support so as to build the pallet load on the pallet    building base;-   a controller operably connected to the at least one articulated    robot and configured to control articulated robot motion, relative    to the pallet building base, and effect therewith the building of a    pallet load build corresponding to the pallet load;-   at least one three-dimensional, time of flight, camera disposed so    as to generate three-dimensional imaging of the pallet load build on    the pallet support on the pallet building base;-   wherein the at least one three-dimensional camera is communicably    coupled to the controller so the controller registers, from the    three-dimensional camera, real time three-dimensional imaging data    embodying different corresponding three-dimensional images of each    different one of the pallet load article units, of the (building)    pallet load build, and-   the controller is configured so as to determine, in real time, from    the corresponding real time three-dimensional imaging data, a build    pallet load variance with respect to a predetermined reference, the    build pallet load variance including identifying at least one of a    presence of an extraneous object in the pallet load build and of a    mispresence (i.e., mispresence is an erroneous position, orientation    or missing presence of the article unit) of at least one pallet load    article unit from the pallet load build; and-   the controller is configured so as to generate in real time an    articulated robot motion signal dependent on the real time    determined build pallet load variance, and the articulated robot    motion signal being generated real time so as to be performed real    time by the articulated robot substantially continuously building    the pallet load build substantially coincident with imaging of the    pallet load build, between placement, by the articulated robot, of    serially consecutive pallet load article units, placed immediately    prior and immediately after imaging of the pallet load build showing    the determined build pallet load variance.

In accordance with one or more aspects of the disclosed embodiment theat least one three-dimensional camera is configured so as to effectthree-dimensional imaging of the pallet support on the pallet buildingbase and of the pallet load build on the pallet support with the atleast one articulated robot effecting substantially continuouspick/place cycles from an input station and placing each of the palletload article units building the pallet load on the pallet building base.

In accordance with one or more aspects of the disclosed embodiment theat least one three-dimensional camera is configured so as to effectthree-dimensional imaging of each respective pallet load article unitsubstantially coincident with placement of the respective pallet loadarticle unit by the at least one articulated robot effectingsubstantially continuous pick/place cycles from an input station andplacing the pallet load article unit building the pallet load buildsubstantially continuously.

In accordance with one or more aspects of the disclosed embodiment theat least one articulated robot motion signal generated by the controlleris a stop motion signal along a pick/place path of the at least onearticulated robot, a slow motion signal along the pick/place path of theat least one articulated robot, or a move to a safe position along safestop path of the at least one articulated robot, different from thepick/place path.

In accordance with one or more aspects of the disclosed embodiment theat least one articulated robot motion signal generated by the controlleris a place position signal setting a place position of at least anotherpallet load article unit.

In accordance with one or more aspects of the disclosed embodiment thepredetermined reference includes a predetermined pallet supportinspection reference defining a predetermined pallet support structurereference characteristic.

In accordance with one or more aspects of the disclosed embodiment thedetermined build pallet load variance includes a pallet support variancethat is a difference determined by the controller between thepredetermined pallet support structure reference characteristic and acharacteristic of the pallet support, imaged by the at least onethree-dimensional camera, corresponding thereto resolved in real time bythe controller from the three-dimensional imaging data.

In accordance with one or more aspects of the disclosed embodiment thecontroller is configured to compare the determined build pallet loadvariance with a predetermined threshold for at least one predeterminedpallet support structure reference characteristic, generate anarticulated robot motion signal (commanding articulated robot stopand/or changing a articulated robot motion path and/or trajectory) ifthe determined build pallet load variance is greater than thepredetermined threshold, and if the determined build pallet loadvariance is less than the predetermined threshold, generate an articleunit place position signal identifying placement of at least anotherpallet load article unit building the pallet load build to the at leastone articulated robot.

In accordance with one or more aspects of the disclosed embodiment thecontroller is configured to set a pallet support base datum of thepallet support, imaged by the at least one three-dimensional camera,from the pallet support variance, which pallet support base datumresolves local base surface variance at each different article unitplace location on the pallet support and defines a real time localarticle unit position base reference for articulated robot placement ofthe at least one article unit of a base article unit layer of palletload build.

In accordance with one or more aspects of the disclosed embodiment thepallet support base datum defines base planarity of the pallet support,and the controller is configured to send a signal to a user, withinformation describing base planarity characteristic, to enableselection of the at least one pallet load article unit of the baselayer, from a number of different size pallet load article units of thepallet load, and of a corresponding placement location on the palletsupport so as to form the base layer based on base planarity.

In accordance with one or more aspects of the disclosed embodiment thebase planarity characteristic information describes planarity variancefor a corresponding area of the base datum in real time, and thecontroller is configured to identify, from the different size palletload article units of the pallet load, one or more pallet load articleunits sized so as to seat stably on the corresponding area so as to formthe base layer.

In accordance with one or more aspects of the disclosed embodiment thepallet support base datum defines base planarity of the pallet support,and the controller is configured to select the at least one pallet loadarticle unit of the base layer, from a number of different size palletload article units of the pallet load, and a corresponding placementlocation on the pallet support so as to form the base layer based onbase planarity.

In accordance with one or more aspects of the disclosed embodiment thecontroller is configured so as to determine in real time, from the realtime three-dimensional imaging data and substantially coincident withsetting of the pallet support base datum, lateral bounds of the palletsupport base datum, wherein at least one of the lateral bounds forms alateral reference datum defining lateral position and orientation of thepallet load build on the pallet load base datum, and forming a referenceframe for placement position of at least one pallet load article unitwith the at least one articulated robot building the pallet load build.

In accordance with one or more aspects of the disclosed embodiment thepredetermined reference includes a predetermined reference position ofthe at least one pallet load article unit in a predetermined referencepallet load build corresponding to the building pallet load build on thepallet support.

In accordance with one or more aspects of the disclosed embodiment thebuild pallet load variance includes an article unit variance that is adifference determined by the controller between a position, resolved inreal time by the controller from the three-dimensional imaging data, ofthe at least one pallet load article unit in the pallet load build andthe predetermined reference position of the at least one pallet loadarticle unit.

In accordance with one or more aspects of the disclosed embodiment apallet building apparatus for user-automatic cooperative building of apallet load of pallet load article units onto a pallet support. Thepallet building apparatus comprises:

-   a frame defining a pallet building base for the pallet support;-   at least one robot connected to the frame and configured so as to    transport and place the pallet load article units serially onto the    pallet support so as to build the pallet load on the pallet building    base;-   a controller operably connected to the at least one robot and    configured to control robot motion, relative to the pallet building    base, and effect therewith the building of the pallet load, and    coupled to a user interface so as to signal a user for cooperation    with the at least one robot effecting building of the pallet load;-   at least one three-dimensional, time of flight, camera disposed so    as to generate three-dimensional imaging of the pallet load build on    the pallet support on the pallet building base;-   wherein the at least one three-dimensional camera is communicably    coupled to the controller so the controller registers, from the at    least one three-dimensional camera, real time three-dimensional    imaging data embodying different corresponding three-dimensional    images of each different one of the pallet load article units, of    the (building) pallet load build, and-   the controller is configured so as to determine, in real time, from    the corresponding real time three-dimensional imaging data, a build    pallet load variance with respect to a predetermined reference, the    build pallet load variance being determinative of at least one of an    extraneous presence, of an extraneous object in the pallet load    build, and of a mispresence of at least one article unit from the    pallet load build; and-   the controller is configured so as to generate in real time a robot    motion signal and a user cooperation signal, both dependent on at    least one of the real time determined build pallet load variance,    the robot motion signal being generated real time so as to be    performed real time by the robot substantially continuously building    the pallet load build substantially coincident with imaging of the    pallet load build, between placement, by the robot, of serially    consecutive pallet load article units, placed immediately prior and    immediately after imaging of the pallet load build showing the    determined build pallet load variance, wherein-   the user cooperation signal defines to the user a deviant condition    of the pallet load build and a cooperative action of the user so as    to resolve the deviant condition depending on the determined at    least one extraneous presence and mispresence.-   In accordance with one or more aspects of the disclosed embodiment    the robot motion signal generated by the controller is a stop motion    signal along a pick/place path of the robot, a slow motion signal    along the pick/place path of the robot, or a move to a safe position    along safe stop path of the robot, different from the pick/place    path.

In accordance with one or more aspects of the disclosed embodiment theuser cooperation signal informs the user of different types of usercooperative action resolving the deviant condition depending on thedetermined at least one extraneous presence and mispresence.

In accordance with one or more aspects of the disclosed embodiment amethod for automatically building a pallet load of pallet load articleunits onto a pallet support is provided. The method comprises:

-   defining, with a frame, a pallet building base for the pallet    support;-   transporting and placing, with at least one articulated robot    connected to the frame, the pallet load article units serially onto    the pallet support so as to build the pallet load on the pallet    building base;-   controlling articulated robot motion, relative to the pallet    building base, with a controller operably connected to the at least    one articulated robot and effecting therewith a pallet load build of    the pallet load;-   generating, with at least one three-dimensional, time of flight,    camera, three-dimensional imaging of the pallet support on the    pallet building base and of the pallet load build on the pallet    support, where the at least one three-dimensional camera is    communicably coupled to the controller;-   registering with the controller, from the at least one    three-dimensional camera, real time three-dimensional imaging data    embodying different corresponding three-dimensional images of the    pallet support and of each different one of the pallet load article    units, and of the (building) pallet load build, and-   determining with the controller, in real time, from the    corresponding real time three-dimensional imaging data, a pallet    support variance or article unit variance of at least one of the    pallet load article units in the pallet load build with respect to a    predetermined reference, and generating in real time an articulated    robot motion signal dependent on at least one of the real time    determined pallet support variance or article unit variance, the    articulated robot motion signal being generated real time so as to    be performed real time by the at least one articulated robot between    placement, by the at least one articulated robot, of at least one    pallet load article unit and a serially consecutive pallet load    article unit enabling substantially continuous building of the    pallet load build.

In accordance with one or more aspects of the disclosed embodiment themethod further comprises three-dimensional imaging, with the at leastone three-dimensional camera, of the pallet support on the palletbuilding base and of the pallet load build on the pallet support withthe at least one articulated robot effecting substantially continuouspick/place cycles from an input station and placing each of the palletload article units building the pallet load on the pallet building base.

In accordance with one or more aspects of the disclosed embodiment themethod further comprises three-dimensional imaging, with the at leastone three-dimensional camera, of each respective pallet load articleunit substantially coincident with placement of the respective palletload article unit by the at least one articulated robot effectingsubstantially continuous pick/place cycles from an input station andplacing the pallet load article unit building the pallet load buildsubstantially continuously.

In accordance with one or more aspects of the disclosed embodiment theat least one articulated robot motion signal generated by the controlleris a stop motion signal along a pick/place path of the at least onearticulated robot, a slow motion signal along the pick/place path of theat least one articulated robot, or a move to a safe position along safestop path of the at least one articulated robot, different from thepick/place path.

In accordance with one or more aspects of the disclosed embodiment thearticulated robot motion signal generated by the controller is a placeposition signal setting a place position of at least another pallet loadarticle unit based on the pallet support variance or the article unitvariance.

In accordance with one or more aspects of the disclosed embodiment thepredetermined reference includes a predetermined pallet supportinspection reference defining a predetermined pallet support structurereference characteristic.

In accordance with one or more aspects of the disclosed embodiment thedetermined pallet support variance is a difference determined by thecontroller between the predetermined pallet support structure referencecharacteristic and a characteristic of the pallet support, imaged by theat least one three-dimensional camera, corresponding thereto resolved inreal time by the controller from the three-dimensional imaging data.

In accordance with one or more aspects of the disclosed embodiment themethod further comprises comparing, with the controller, the determinedpallet support variance with a predetermined threshold for at least onepredetermined pallet support structure reference characteristic,generating an articulated robot motion signal (commanding articulatedrobot stop and/or changing a articulated robot motion path and/ortrajectory) if the determined pallet support variance is greater thanthe predetermined threshold, and if the determined pallet supportvariance is less than the predetermined threshold, generating an articleunit place position signal identifying placement of at least anotherpallet load article unit building the pallet load build to the at leastone articulated robot.

In accordance with one or more aspects of the disclosed embodiment themethod further comprises setting, with the controller, a pallet supportbase datum of the pallet support, imaged by the at least onethree-dimensional camera, from the pallet support variance, which palletsupport base datum resolves local base surface variance at eachdifferent article unit place location on the pallet support and definesa real time local article unit position base reference for articulatedrobot placement of the at least one article unit of a base article unitlayer of pallet load build.

In accordance with one or more aspects of the disclosed embodiment thepallet support base datum defines base planarity of the pallet support,and the method further comprises sending, with the controller, a signalto a user, with information describing base planarity characteristic, toenable selection of the at least one pallet load article unit of thebase layer, from a number of different size pallet load article units ofthe pallet load, and of a corresponding placement location on the palletsupport so as to form the base layer based on base planarity.

In accordance with one or more aspects of the disclosed embodiment thebase planarity characteristic information describes planarity variancefor a corresponding area of the base datum in real time, and the methodfurther comprises identifying with the controller, from the differentsize pallet load article units of the pallet load, one or more palletload article units sized so as to seat stably on the corresponding areaso as to form the base layer.

In accordance with one or more aspects of the disclosed embodiment thepallet support base datum defines base planarity of the pallet support,and the method further comprises selecting, with the controller, the atleast one pallet load article unit of the base layer, from a number ofdifferent size pallet load article units of the pallet load, and acorresponding placement location on the pallet support so as to form thebase layer based on base planarity.

In accordance with one or more aspects of the disclosed embodiment themethod further comprises determining, with the controller in real time,from the real time three-dimensional imaging data and substantiallycoincident with setting of the pallet support base datum, lateral boundsof the pallet support base datum, wherein at least one of the lateralbounds forms a lateral reference datum defining lateral position andorientation of the pallet load build on the pallet load base datum, andforming a reference frame for placement position of the at least onepallet load article unit with the at least one articulated robotbuilding the pallet load build.

In accordance with one or more aspects of the disclosed embodiment thepredetermined reference includes a predetermined reference position ofthe at least one pallet load article unit in a predetermined referencepallet load build corresponding to the building pallet load build on thepallet support.

In accordance with one or more aspects of the disclosed embodiment thedetermined article unit variance is a difference determined by thecontroller between a position, resolved in real time by the controllerfrom the three-dimensional imaging data, of the at least one pallet loadarticle unit in the pallet load build and the predetermined referenceposition of the at least one pallet load article unit.

In accordance with one or more aspects of the disclosed embodiment amethod for automatically building a pallet load of pallet load articleunits onto a pallet support. The method comprises:

-   defining, with a frame, a pallet building base for the pallet    support;-   transporting and placing the pallet load article units, with at    least one articulated robot connected to the frame, serially onto    the pallet support so as to build the pallet load on the pallet    building base;-   controlling, with a controller operably connected to the at least    one articulated robot, articulated robot motion, relative to the    pallet building base, and effecting therewith the building of a    pallet load build corresponding to the pallet load;-   generating, with at least one three-dimensional, time of flight,    camera, three-dimensional imaging of the pallet load build on the    pallet support on the pallet building base where the at least one    three-dimensional camera is communicably coupled to the controller;-   registering with the controller, from the three-dimensional camera,    real time three-dimensional imaging data embodying different    corresponding three-dimensional images of each different one of the    pallet load article units, of the (building) pallet load build, and-   determining, with the controller in real time, from the    corresponding real time three-dimensional imaging data, a build    pallet load variance with respect to a predetermined reference, the    build pallet load variance including identifying at least one of a    presence of an extraneous object in the pallet load build and of a    mispresence (i.e., mispresence is an erroneous position, orientation    or missing presence of the article unit) of at least one pallet load    article unit from the pallet load build; and-   generating, with the controller, in real time an articulated robot    motion signal dependent on at least one of the real time determined    build pallet load variance, and the articulated robot motion signal    being generated real time so as to be performed real time by the    articulated robot substantially continuously building the pallet    load build substantially coincident with imaging of the pallet load    build, between placement, by the articulated robot, of serially    consecutive pallet load article units, placed immediately prior and    immediately after imaging of the pallet load build showing the    determined build pallet load variance.

In accordance with one or more aspects of the disclosed embodiment themethod further comprises three-dimensional imaging, with the at leastone three-dimensional camera, of the pallet support on the palletbuilding base and of the pallet load build on the pallet support withthe at least one articulated robot effecting substantially continuouspick/place cycles from an input station and placing each of the palletload article units building the pallet load on the pallet building base.

In accordance with one or more aspects of the disclosed embodiment themethod further comprises three-dimensional imaging, with the at leastone three-dimensional camera, of each respective pallet load articleunit substantially coincident with placement of the respective palletload article unit by the at least one articulated robot effectingsubstantially continuous pick/place cycles from an input station andplacing the pallet load article unit building the pallet load buildsubstantially continuously.

In accordance with one or more aspects of the disclosed embodiment theat least one articulated robot motion signal generated by the controlleris a stop motion signal along a pick/place path of the at least onearticulated robot, a slow motion signal along the pick/place path of theat least one articulated robot, or a move to a safe position along safestop path of the at least one articulated robot, different from thepick/place path.

In accordance with one or more aspects of the disclosed embodiment theat least one articulated robot motion signal generated by the controlleris a place position signal setting a place position of at least anotherpallet load article unit.

In accordance with one or more aspects of the disclosed embodiment thepredetermined reference includes a predetermined pallet supportinspection reference defining a predetermined pallet support structurereference characteristic.

In accordance with one or more aspects of the disclosed embodiment thedetermined build pallet load variance includes a pallet support variancethat is a difference determined by the controller between thepredetermined pallet support structure reference characteristic and acharacteristic of the pallet support, imaged by the at least onethree-dimensional camera, corresponding thereto resolved in real time bythe controller from the three-dimensional imaging data.

In accordance with one or more aspects of the disclosed embodiment themethod further comprises comparing, with the controller, the determinedbuild pallet load variance with a predetermined threshold for at leastone predetermined pallet support structure reference characteristic,generating an articulated robot motion signal (commanding articulatedrobot stop and/or changing a articulated robot motion path and/ortrajectory) if the determined build pallet load variance is greater thanthe predetermined threshold, and if the determined build pallet loadvariance is less than the predetermined threshold, generating an articleunit place position signal identifying placement of at least anotherpallet load article unit building the pallet load build to the at leastone articulated robot.

In accordance with one or more aspects of the disclosed embodiment themethod further comprises setting, with the controller, a pallet supportbase datum of the pallet support, imaged by the at least onethree-dimensional camera, from the pallet support variance, which palletsupport base datum resolves local base surface variance at eachdifferent article unit place location on the pallet support and definesa real time local article unit position base reference for articulatedrobot placement of the at least one article unit of a base article unitlayer of pallet load build.

In accordance with one or more aspects of the disclosed embodiment thepallet support base datum defines base planarity of the pallet support,and the method further comprises sending, with the controller, a signalto a user, with information describing base planarity characteristic, toenable selection of the at least one pallet load article unit of thebase layer, from a number of different size pallet load article units ofthe pallet load, and of a corresponding placement location on the palletsupport so as to form the base layer based on base planarity.

In accordance with one or more aspects of the disclosed embodiment thebase planarity characteristic information describes planarity variancefor a corresponding area of the base datum in real time, and the methodfurther comprises identifying with the controller, from the differentsize pallet load article units of the pallet load, one or more palletload article units sized so as to seat stably on the corresponding areaso as to form the base layer.

In accordance with one or more aspects of the disclosed embodiment thepallet support base datum defines base planarity of the pallet support,and the method further comprises selecting, with the controller, the atleast one pallet load article unit of the base layer, from a number ofdifferent size pallet load article units of the pallet load, and acorresponding placement location on the pallet support so as to form thebase layer based on base planarity.

In accordance with one or more aspects of the disclosed embodiment themethod further comprises determining, with the controller in real time,from the real time three-dimensional imaging data and substantiallycoincident with setting of the pallet support base datum, lateral boundsof the pallet support base datum, wherein at least one of the lateralbounds forms a lateral reference datum defining lateral position andorientation of the pallet load build on the pallet load base datum, andforming a reference frame for placement position of at least one palletload article unit with the at least one articulated robot building thepallet load build.

In accordance with one or more aspects of the disclosed embodiment thepredetermined reference includes a predetermined reference position ofthe at least one pallet load article unit in a predetermined referencepallet load build corresponding to the building pallet load build on thepallet support.

In accordance with one or more aspects of the disclosed embodiment thebuild pallet load variance includes an article unit variance that is adifference determined by the controller between a position, resolved inreal time by the controller from the three-dimensional imaging data, ofthe at least one pallet load article unit in the pallet load build andthe predetermined reference position of the at least one pallet loadarticle unit.

In accordance with one or more aspects of the disclosed embodiment amethod for user-automatic cooperative building of a pallet load ofpallet load article units onto a pallet support is provided. The methodcomprises:

-   defining, with a frame, a pallet building base for the pallet    support;-   transporting and placing, with at least one robot connected to the    frame, the pallet load article units serially onto the pallet    support so as to build the pallet load on the pallet building base;-   controlling, with a controller operably connected to the at least    one robot, robot motion, relative to the pallet building base, and    effecting therewith the building of the pallet load, and signaling,    with a user interface coupled to the controller, a user for    cooperation with the at least one robot effecting building of the    pallet load;-   generating, with at least one three-dimensional, time of flight,    camera, three-dimensional imaging of the pallet load build on the    pallet support on the pallet building base, where the at least one    three-dimensional camera is communicably coupled to the controller;-   registering with the controller, from the at least one    three-dimensional camera, real time three-dimensional imaging data    embodying different corresponding three-dimensional images of each    different one of the pallet load article units, of the (building)    pallet load build, and-   determining with the controller, in real time, from the    corresponding real time three-dimensional imaging data, a build    pallet load variance with respect to a predetermined reference, the    build pallet load variance being determinative of at least one of an    extraneous presence, of an extraneous object in the pallet load    build, and of a mispresence of at least one article unit from the    pallet load build; and-   generating, with the controller, in real time a robot motion signal    and a user cooperation signal, both dependent on at least one of the    real time determined build pallet load variance, the robot motion    signal being generated real time so as to be performed real time by    the robot substantially continuously building the pallet load build    substantially coincident with imaging of the pallet load build,    between placement, by the robot, of serially consecutive pallet load    article units, placed immediately prior and immediately after    imaging of the pallet load build showing the determined build pallet    load variance, wherein-   the user cooperation signal defines to the user a deviant condition    of the pallet load build and a cooperative action of the user so as    to resolve the deviant condition depending on the determined at    least one extraneous presence and mispresence.

In accordance with one or more aspects of the disclosed embodiment therobot motion signal generated by the controller is a stop motion signalalong a pick/place path of the robot, a slow motion signal along thepick/place path of the robot, or a move to a safe position along safestop path of the robot, different from the pick/place path.

In accordance with one or more aspects of the disclosed embodiment theuser cooperation signal informs the user of different types of usercooperative action resolving the deviant condition depending on thedetermined at least one extraneous presence and mispresence.

It should be understood that the foregoing description is onlyillustrative of the aspects of the disclosed embodiment. Variousalternatives and modifications can be devised by those skilled in theart without departing from the aspects of the disclosed embodiment.Accordingly, the aspects of the disclosed embodiment are intended toembrace all such alternatives, modifications and variances that fallwithin the scope of the appended claims. Further, the mere fact thatdifferent features are recited in mutually different dependent orindependent claims does not indicate that a combination of thesefeatures cannot be advantageously used, such a combination remainingwithin the scope of the aspects of the invention.

What is claimed is:
 1. A method for automatically building a pallet loadof pallet load article units onto a pallet support, the methodcomprising: defining, with a frame, a pallet building base for thepallet support; transporting and placing the pallet load article units,with at least one articulated robot connected to the frame, seriallyonto the pallet support so as to build the pallet load on the palletbuilding base; controlling, with a controller operably connected to theat least one articulated robot, articulated robot motion, relative tothe pallet building base, and effecting therewith the building of apallet load build corresponding to the pallet load; generating, with atleast one three-dimensional, time of flight camera, three-dimensionalimaging of the pallet load build on the pallet support on the palletbuilding base, wherein the at least one three-dimensional camera iscommunicably coupled to the controller; registering with the controller,from the three-dimensional camera, real time three-dimensional imagingdata embodying different corresponding three-dimensional images of eachdifferent one of the pallet load article units, of the pallet loadbuild, and determining, with the controller in real time, from thecorresponding real time three-dimensional imaging data, a build palletload variance with respect to a predetermined reference, the buildpallet load variance including identifying at least one of a presence ofan extraneous object in the pallet load build and of a mispresence of atleast one pallet load article unit from the pallet load build; andgenerating, with the controller, in real time an articulated robotmotion signal dependent on at least one of the real time determinedbuild pallet load variance, the articulated robot motion signal beinggenerated in real time so as to be performed in real time by thearticulated robot building the pallet load build substantiallycoincident with the imaging of the pallet load build by the articulatedrobot, wherein the pallet load build is substantially continuously builtby substantially continuous placement of serially consecutive palletload article units, wherein the consecutive pallet load article unitsare placed at least one of immediately prior and immediately after theimaging of the pallet load build showing the determined build palletload variance.
 2. The method of claim 1, further comprisingthree-dimensional imaging, with the at least one three-dimensionalcamera, of the pallet support on the pallet building base and of thepallet load build on the pallet support with the at least onearticulated robot effecting substantially continuous pick/place cyclesfrom an input station and placing each of the pallet load article unitsbuilding the pallet load on the pallet building base.
 3. The method ofclaim 1, further comprising three-dimensional imaging, with the at leastone three-dimensional camera, of each respective pallet load articleunit substantially coincident with placement of the respective palletload article unit by the at least one articulated robot effectingsubstantially continuous pick/place cycles from an input station andplacing the pallet load article unit building the pallet load buildsubstantially continuously.
 4. The method of claim 1, wherein the atleast one articulated robot motion signal generated by the controller isa stop motion signal along a pick/place path of the at least onearticulated robot, a slow motion signal along the pick/place path of theat least one articulated robot, or a move to a safe position along safestop path of the at least one articulated robot, different from thepick/place path.
 5. The method of claim 1, wherein the at least onearticulated robot motion signal generated by the controller is a placeposition signal setting a place position of at least another pallet loadarticle unit.
 6. The method of claim 5, wherein the predeterminedreference includes a predetermined pallet support inspection referencedefining a predetermined pallet support structure referencecharacteristic.
 7. The method of claim 6, wherein the determined buildpallet load variance includes a pallet support variance that is adifference determined by the controller between the predetermined palletsupport structure reference characteristic and a characteristic of thepallet support, imaged by the at least one three-dimensional camera,corresponding thereto resolved in real time by the controller from thethree-dimensional imaging data.
 8. The method of claim 1, furthercomprising one or more of: comparing, with the controller, thedetermined build pallet load variance with a predetermined threshold forat least one predetermined pallet support structure referencecharacteristic, generating an articulated robot motion signal commandingarticulated robot stop, and changing one or more of an articulated robotmotion path and an articulated robot trajectory if the determined buildpallet load variance is greater than the predetermined threshold, and ifthe determined build pallet load variance is less than the predeterminedthreshold, generating an article unit place position signal identifyingplacement of at least another pallet load article unit building thepallet load build to the at least one articulated robot.
 9. The methodof claim 1, further comprising setting, with the controller, a palletsupport base datum of the pallet support, imaged by the at least onethree-dimensional camera, from the pallet support variance, which palletsupport base datum resolves local base surface variance at eachdifferent article unit place location on the pallet support, and definesa real time local article unit position base reference for articulatedrobot placement of the at least one article unit of a base article unitlayer of the pallet load build.
 10. The method of claim 9, wherein thepallet support base datum defines base planarity of the pallet support,and the method further comprises sending, with the controller, a signalto a user, with information describing base planarity characteristic, toenable selection of the at least one pallet load article unit of thebase layer, from a number of different size pallet load article units ofthe pallet load, and of a corresponding placement location on the palletsupport so as to form the base layer based on base planarity.
 11. Themethod of claim 10, wherein the base planarity characteristicinformation describes planarity variance for a corresponding area of thebase datum in real time, and the method further comprises identifyingwith the controller, from the different size pallet load article unitsof the pallet load, one or more pallet load article units sized so as toseat stably on the corresponding area so as to form the base layer. 12.The method of claim 9, wherein the pallet support base datum definesbase planarity of the pallet support, and the method further comprisesselecting, with the controller, the at least one pallet load articleunit of the base layer, from a number of different size pallet loadarticle units of the pallet load, and a corresponding placement locationon the pallet support so as to form the base layer based on baseplanarity.
 13. The method of claim 9, further comprising determining,with the controller in real time, from the real time three-dimensionalimaging data and substantially coincident with setting of the palletsupport base datum, lateral bounds of the pallet support base datum,wherein at least one of the lateral bounds forms a lateral referencedatum defining lateral position and orientation of the pallet load buildon the pallet load base datum, and forming a reference frame forplacement position of at least one pallet load article unit with the atleast one articulated robot building the pallet load build.
 14. Themethod of claim 1, wherein the predetermined reference includes apredetermined reference position of the at least one pallet load articleunit in a predetermined reference pallet load build corresponding to thebuilding pallet load build on the pallet support.
 15. The method ofclaim 1, wherein the build pallet load variance includes an article unitvariance that is a difference determined by the controller between aposition, resolved in real time by the controller from thethree-dimensional imaging data, of the at least one pallet load articleunit in the pallet load build and the predetermined reference positionof the at least one pallet load article unit.
 16. A method foruser-automatic cooperative building of a pallet load of pallet loadarticle units onto a pallet support, the method comprising: defining,with a frame, a pallet building base for the pallet support;transporting and placing, with at least one robot connected to theframe, the pallet load article units serially onto the pallet support soas to build the pallet load on the pallet building base; controlling,with a controller operably connected to the at least one robot, robotmotion, relative to the pallet building base, and effecting therewiththe building of the pallet load, and signaling, with a user interfacecoupled to the controller, a user for cooperation with the at least onerobot effecting building of the pallet load; generating, with at leastone three-dimensional, time of flight camera, three-dimensional imagingof the pallet load build on the pallet support on the pallet buildingbase, wherein the at least one three-dimensional camera is communicablycoupled to the controller; registering with the controller, from the atleast one three-dimensional camera, real time three-dimensional imagingdata embodying different corresponding three-dimensional images of eachdifferent one of the pallet load article units, of the pallet loadbuild, and determining with the controller, in real time, from thecorresponding real time three-dimensional imaging data, a build palletload variance with respect to a predetermined reference, the buildpallet load variance being determinative of at least one of anextraneous presence, of an extraneous object in the pallet load build,and of a mispresence of at least one article unit from the pallet loadbuild; and generating, with the controller, in real time a robot motionsignal and a user cooperation signal, both dependent on at least one ofthe real time determined build pallet load variance, the robot motionsignal being generated in real time so as to be performed in real timeby the robot building the pallet load build substantially coincidentwith the imaging of the pallet load build by the robot, wherein thepallet load build is built by substantially continuous placement ofserially consecutive pallet load article units, wherein the consecutivepallet load article units are placed immediately prior or immediatelyafter the imaging of the pallet load build showing the determined buildpallet load variance, wherein the user cooperation signal defines to theuser a deviant condition of the pallet load build and a cooperativeaction of the user so as to resolve the deviant condition depending onthe determined at least one extraneous presence and mispresence.
 17. Themethod of claim 16, wherein the robot motion signal generated by thecontroller is a stop motion signal along a pick/place path of the robot,a slow motion signal along the pick/place path of the robot, or a moveto a safe position along safe stop path of the robot, different from thepick/place path.
 18. The method of claim 16, wherein the usercooperation signal informs the user of different types of usercooperative action resolving the deviant condition depending on thedetermined at least one extraneous presence and mispresence.
 19. Themethod of claim 16, further comprising one or more of: comparing, withthe controller, the determined build pallet load variance with apredetermined threshold for at least one predetermined pallet supportstructure reference characteristic, generating an articulated robotmotion signal commanding articulated robot stop, and changing one ormore of an articulated robot motion path and an articulated robottrajectory if the determined build pallet load variance is greater thanthe predetermined threshold, and if the determined build pallet loadvariance is less than the predetermined threshold, generating an articleunit place position signal identifying placement of at least anotherpallet load article unit building the pallet load build to the at leastone articulated robot.
 20. The method of claim 16, further comprisingsetting, with the controller, a pallet support base datum of the palletsupport, imaged by the at least one three-dimensional camera, from thepallet support variance, which pallet support base datum resolves localbase surface variance at each different article unit place location onthe pallet support, and defines a real time local article unit positionbase reference for articulated robot placement of the at least onearticle unit of a base article unit layer of the pallet load build. 21.A pallet load builder for automatically building pallet load articleunits onto a pallet support, the pallet load builder comprising: a framedefining a pallet building base for the pallet support; at least onearticulated robot connected to the frame and configured for transportingand placing the pallet load article units serially onto the palletsupport so as to build the pallet load on the pallet building base; acontroller configured to effect the serially loading of the pallet loadarticle units onto the pallet support so as to build the pallet load,wherein the controller effects generation, with at least onethree-dimensional, time of flight camera, of three-dimensional imagingof the pallet load build on the pallet support on the pallet buildingbase and registration of real time three-dimensional imaging dataembodying different corresponding three-dimensional images of eachdifferent one of the pallet load article units, of the pallet loadbuild, so as to determine, in real time, from the corresponding realtime three-dimensional imaging data, a build pallet load variance withrespect to a predetermined reference, the build pallet load varianceincluding identifying at least one of a presence of an extraneous objectin the pallet load build and of a mispresence of at least one palletload article unit from the pallet load build, and wherein the controllergenerates, in real time, an articulated robot motion signal dependent onat least one of the real time determined build pallet load variance, thearticulated robot motion signal being generated in real time so as to beperformed in real time by the articulated robot building the pallet loadbuild substantially coincident with the imaging of the pallet load buildby the articulated robot, wherein the pallet load build is substantiallycontinuously built by substantially continuous placement of seriallyconsecutive pallet load article units, wherein the consecutive palletload article units are placed at least one of immediately prior andimmediately after the imaging of the pallet load build showing thedetermined build pallet load variance.
 22. The pallet load builder ofclaim 21, wherein the at least one three-dimensional camera isconfigured to three-dimensional image the pallet support on the palletbuilding base and of the pallet load build on the pallet support withthe at least one articulated robot effecting substantially continuouspick/place cycles from an input station and placing each of the palletload article units building the pallet load on the pallet building base.23. The pallet load builder of claim 21, wherein the at least onethree-dimensional camera is configured to three-dimensional image eachrespective pallet load article unit substantially coincident withplacement of the respective pallet load article unit by the at least onearticulated robot effecting substantially continuous pick/place cyclesfrom an input station and placing the pallet load article unit buildingthe pallet load build substantially continuously.
 24. The pallet loadbuilder of claim 21, wherein the at least one articulated robot motionsignal generated by the controller is a stop motion signal along apick/place path of the at least one articulated robot, a slow motionsignal along the pick/place path of the at least one articulated robot,or a move to a safe position along safe stop path of the at least onearticulated robot, different from the pick/place path.
 25. The palletload builder of claim 21, wherein the at least one articulated robotmotion signal generated by the controller is a place position signalsetting a place position of at least another pallet load article unit.26. The pallet load builder of claim 25, wherein the predeterminedreference includes a predetermined pallet support inspection referencedefining a predetermined pallet support structure referencecharacteristic.
 27. The pallet load builder of claim 26, wherein thedetermined build pallet load variance includes a pallet support variancethat is a difference determined by the controller between thepredetermined pallet support structure reference characteristic and acharacteristic of the pallet support, imaged by the at least onethree-dimensional camera, corresponding thereto resolved in real time bythe controller from the three-dimensional imaging data.
 28. The palletload builder of claim 21, wherein the controller is further configuredto one or more of: compare the determined build pallet load variancewith a predetermined threshold for at least one predetermined palletsupport structure reference characteristic, generate an articulatedrobot motion signal commanding articulated robot stop, and change one ormore of an articulated robot motion path and an articulated robottrajectory if the determined build pallet load variance is greater thanthe predetermined threshold, and if the determined build pallet loadvariance is less than the predetermined threshold, generating an articleunit place position signal identifying placement of at least anotherpallet load article unit building the pallet load build to the at leastone articulated robot.
 29. The pallet load builder of claim 21, whereinthe controller is configured to set a pallet support base datum of thepallet support, imaged by the at least one three-dimensional camera,from the pallet support variance, which pallet support base datumresolves local base surface variance at each different article unitplace location on the pallet support, and defines a real time localarticle unit position base reference for articulated robot placement ofthe at least one article unit of a base article unit layer of the palletload build.
 30. The pallet load builder of claim 29, wherein the palletsupport base datum defines base planarity of the pallet support, and thecontroller is configured to send a signal to a user, with informationdescribing base planarity characteristic, to enable selection of the atleast one pallet load article unit of the base layer, from a number ofdifferent size pallet load article units of the pallet load, and of acorresponding placement location on the pallet support so as to form thebase layer based on base planarity.
 31. The pallet load builder of claim30, wherein the base planarity characteristic information describesplanarity variance for a corresponding area of the base datum in realtime, and the controller is configured to identify, from the differentsize pallet load article units of the pallet load, one or more palletload article units sized so as to seat stably on the corresponding areaso as to form the base layer.
 32. The pallet load builder of claim 29,wherein the pallet support base datum defines base planarity of thepallet support, and the controller is configured to select the at leastone pallet load article unit of the base layer, from a number ofdifferent size pallet load article units of the pallet load, and acorresponding placement location on the pallet support so as to form thebase layer based on base planarity.
 33. The pallet load builder of claim29, wherein the controller is configured to determine, in real time,from the real time three-dimensional imaging data and substantiallycoincident with setting of the pallet support base datum, lateral boundsof the pallet support base datum, wherein at least one of the lateralbounds forms a lateral reference datum defining lateral position andorientation of the pallet load build on the pallet load base datum, andforming a reference frame for placement position of at least one palletload article unit with the at least one articulated robot building thepallet load build.
 34. The pallet load builder of claim 21, wherein thepredetermined reference includes a predetermined reference position ofthe at least one pallet load article unit in a predetermined referencepallet load build corresponding to the building pallet load build on thepallet support.
 35. The pallet load builder of claim 21, wherein thebuild pallet load variance includes an article unit variance that is adifference determined by the controller between a position, resolved inreal time by the controller from the three-dimensional imaging data, ofthe at least one pallet load article unit in the pallet load build andthe predetermined reference position of the at least one pallet loadarticle unit.
 36. A pallet load builder for user-automatic cooperativebuilding of pallet load article units onto a pallet support, the palletload builder comprising: a frame defining a pallet building base for thepallet support; at least one robot connected to the frame and configuredto transport and place the pallet load article units serially onto thepallet support so as to build the pallet load on the pallet buildingbase; and a controller configured to control the at least one robot androbot motion and effect therewith the serially loading of the palletload, and signal, with a user interface coupled to the controller, auser for cooperation with the at least one robot effecting building ofthe pallet load, wherein the controller is further configured togenerate, with at least one three-dimensional, time of flight camera,three-dimensional imaging of the pallet load build on the pallet supporton the pallet building base, wherein the at least one three-dimensionalcamera is communicably coupled to the controller, register, from the atleast one three-dimensional camera, real time three-dimensional imagingdata embodying different corresponding three-dimensional images of eachdifferent one of the pallet load article units, of the pallet loadbuild, and determine, in real time, from the corresponding real timethree-dimensional imaging data, a build pallet load variance withrespect to a predetermined reference, the build pallet load variancebeing determinative of at least one of an extraneous presence, of anextraneous object in the pallet load build, and of a mispresence of atleast one article unit from the pallet load build, and wherein thecontroller generates, in real time a robot motion signal and a usercooperation signal, both dependent on at least one of the real timedetermined build pallet load variance, the robot motion signal beinggenerated in real time so as to be performed in real time by the robotbuilding the pallet load build substantially coincident with the imagingof the pallet load build by the robot, wherein the pallet load build isbuilt by substantially continuous placement of serially consecutivepallet load article units, wherein the consecutive pallet load articleunits are placed immediately prior or immediately after the imaging ofthe pallet load build showing the determined build pallet load variance,wherein the user cooperation signal defines to the user a deviantcondition of the pallet load build and a cooperative action of the userso as to resolve the deviant condition depending on the determined atleast one extraneous presence and mispresence.
 37. The pallet loadbuilder of claim 36, wherein the robot motion signal generated by thecontroller is a stop motion signal along a pick/place path of the robot,a slow motion signal along the pick/place path of the robot, or a moveto a safe position along safe stop path of the robot, different from thepick/place path.
 38. The pallet load builder of claim 36, wherein theuser cooperation signal informs the user of different types of usercooperative action resolving the deviant condition depending on thedetermined at least one extraneous presence and mispresence.
 39. Thepallet load builder of claim 36, wherein the controller is configured toone or more of: compare the determined build pallet load variance with apredetermined threshold for at least one predetermined pallet supportstructure reference characteristic, generate an articulated robot motionsignal commanding articulated robot stop, and change one or more of anarticulated robot motion path and an articulated robot trajectory if thedetermined build pallet load variance is greater than the predeterminedthreshold, and if the determined build pallet load variance is less thanthe predetermined threshold, generating an article unit place positionsignal identifying placement of at least another pallet load articleunit building the pallet load build to the at least one articulatedrobot.
 40. The pallet load builder of claim 36, wherein the controlleris configured to set a pallet support base datum of the pallet support,imaged by the at least one three-dimensional camera, from the palletsupport variance, which pallet support base datum resolves local basesurface variance at each different article unit place location on thepallet support, and defines a real time local article unit position basereference for articulated robot placement of the at least one articleunit of a base article unit layer of the pallet load build.